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C. Michele Matz. Administration of Web Versus Paper Surveys: Mode Effects and 
Response Rates. A Master's paper for the M.S. in L.S. degree. November, 1999. 86 
pages. Advisor: Barbara M. Wildemuth. 

A survey of academic reference librarians in North Carolina provided data for an 
examination of differences in survey administration on paper and the World Wide Web. 
Research via the Internet is becoming more attractive for many researchers, but the 
effects of this medium upon research outcomes has been little explored. This study 
examined in particular sampling and mode effects, and response rates of Web surveys. 

The study found no sampling bias or mode effects in tests of the respondents’ 
demographics and the content of responses. Response rates to Web surveys are not as 
high as traditional survey methods, and while responses are gathered more quickly, the 
paper instrument was not far behind. Email notices were more efficient for promoting 
the Web survey than paper notices. Traditional postal surveys still hold some advantages 
over Web surveys. Researchers must weigh the advantages in cost and speed to justify 
use of such instruments. 

Headings: 

S urveys — methodol ogy 
S urveys — eval uati on 
Surveys — Internet 
College and university libraries 
Surveys — reference services 

Library schools - Theses - University of North Carolina at Chapel Hill 







3 



i 



3 



Table of Contents 



4 

I. Introduction 

5 

II. Literature Review 6 

Sampling Effects 7 

Mode Effects 1 1 

Advantages of Electronic Surveys 

Publicizing Electronic Surveys 16 

Summary of Research Questions 

17 

III. Methodology ' ig 

Population 19 

The Survey 20 

Procedures for Distributing the Surveys 

22 

IV. Results 23 

Hypotheses I 23 

Hypotheses 2 and 3 25 

Hypothesis 4 25 

Hypothesis 5 27 

Hypothesis 6 

28 

V. Discussion 30 



Technical Difficulties 

31 

VI. Conclusion 

33 



Bibliography 

A-I 

Appendix A - Paper Survey g_l 

Appendix B - Web Survey C-I 

Appendix C — Cover Letters 

Appendix D - Selected Statistical Test Results ...............D-I 

Tests for Hypothesis I j )_5 

Tests for Hypotheses 2 and 3 D-I 3 

Tests for Hypothesis 5 D-I4 

Other Comparisons g.l 

Appendix E - Survey Content Summary 




’V- 



* 



4 



I. Introduction to the problem 

Social science researchers have noted that the medium in which a researcher 
gathers data may affect the data gathered (Babbie, 1998). This effect may skew what 
audience is reached, the kinds of information gathered or whether the audience self- 
selects in a particular way. A number of studies have examined the benefits and 
liabilities of various data collection methods including personal interviews, telephone 
interviews, mail surveys, and electronic mail surveys; however, few studies have 
examined surveys administered on the World Wide Web. While a growing number of 
surveys are being posted on Web sites and there is much speculation as to how this new 
medium will affect the results, little outcome data is available. 

Whereas surveys have been administered electronically — that is, on computers — 
since the late 1960s, these were usually surveys that were given to traditional population 
samples. Participants were invited to go to a particular place where they would sit at a 
dumb terminal and answer the survey questions as they were prompted by the screen. A 
researcher might or might not be present to clarify questions. Electronic mail offered a 
way to send the survey to the respondent to answer at his or her convenience. A number 
of the earlier studies of email surveys were conducted within a particular organization: a 
university or a company. Very quickly, though, researchers began to see the potential of 
reaching a much broader audience via such avenues as listservs. Here was a way to 
negate geographic boundaries and reach very large numbers of people. Web surveys 
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seem to offer many of the same benefits as email surveys, but with a much nicer graphic 
interface, and electronic forms (with form “objects”) that provide the means for a 
researcher to standardize the responses. (For example, radio buttons will only permit one 

answer, while check boxes allow multiple responses.) 

Many studies have established the skew in demographics of Internet users (e.g., 
the GVU survey [GVU Center, 1998]). The current study sought to examine directly this 
sampling effect. Specifically, it controlled for sampling bias by surveying a population 
that, while Intemet-savvy, is not circumscribed by participation in the Internet. In 
addition, the responses received via a Web survey were compared with those received via 
a traditional paper survey. Finally, two variations in promoting the Web survey were 

compared; a letter mailed in the usual way and an email letter. 

' Although Web surveys are only beginning to be examined in the literature, studies 

of electronic mail surveys have demonstrated some administrative benefits that should b 
equally applicable for Web surveys. Email surveys have been shown to be particularly 
advantageous in terms of cost and speed. Further questions for the study involved how 
quickly surveys were returned, and the comparative response rates elicited by the Web 

and paper instruments. 

II. Literature review 

Sample effects of Internet populations and mode effects of electronic surveys are 
major factors for researchers to consider when conducting a survey with an Internet 
population. It is important to understand the implications of such effects in order to avoid 
their consequences, as this study attempts to do. In particular, the sampling effects and 
mode effects that occurred with the use of electronic mail surveys will be discussed, as 
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well as the ways in which such effects had an impact on study results. In spite of the 
disadvantages associated with sampling and mode effects, there are distinct advantages to 
administering surveys electronically, both by email and over the Web. These advantages 
will be described along with other characteristics of the two types of surveys. 

Implications for the current study will be considered. 

Sampling Effects 

A major factor affecting Web surveys is the overall population that uses the 
Internet has different characteristics than the general population. A recent survey of 
Internet users found 67.5% of their respondents were men (Pitkow, 1996). Other groups 
over-represented among Internet users are whites, the young, the rich, and the highly 
educated (GVU Center, 1998; Anderson and Gansneder, 1995). If the sampling frame 
from which a researcher selects a study population is unrepresentative of the general 
population, that study will exhibit the skew of the sample. Because Internet users do not 
constitute a representative sample of the population, researchers have been wary of the 
potential for a strong sampling bias (Shaw and Davis, 1996; Walsh, Kiesler, Sproull and 
Hesse, 1992). This continues to be true, although some of these same studies 
demonstrate the speed with which Internet use is becoming mainstream (Pitkow, 1996; 
GVU Center, 1998). 

The skew in the demographics of the Internet population versus the general public 
clearly impacts what kind of research can be conducted via the Web, and the kinds of 
generalizations one can make from data collected in this manner. And although more 
research is being conducted with groups that are clearly Internet-literate, further questions 
must be asked, such as whether members of such a group have equal access to the 
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Internet and use it in comparable ways and at comparable levels. When the answer to 
any of these questions is no, predicting how survey responses will consequently be 
distorted becomes very difficult. 

Mode Effects 

One of the earliest surveys of electronic research identified significant mode 

effects differences in results caused by the medium in which the survey was 

administered. Kiesler and Sproull (1986) queried students and faculty of a major 
academic institution who were known to have used email recently. They found important 
differences between the responses to their email versus print surveys. 

Electronic respondents were more cooperative, returning a larger number of 
surveys in a shorter period of time than paper respondents. Electronic respondents made 
fewer errors in responding to questions and refused to answer or skipped fewer questions 
than paper respondents. And finally, the electronic responses were more "extreme," or 
further from a socially accepted norm. They theorized that the lack of social context in 
the electronic medium, normally provided by such cues as a cover letter’s institutional 
letterhead, resulted in respondents feeling less inhibited to respond freely. 

Kiesler and Sproull's results were further strengthened when they re-admmistered 
the survey four months after the initial instrument to volunteers from the original group 
of respondents. They switched the medium in which the subjects received the survey to 
the one each group had not used in the first round. Although the number of responses 
was smaller, the anticipated effects were the same. They concluded that, although there 
was "considerable similarity of response between the paper and electronic survey", it was 
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"not so much that the two may be considered interchangeable without further research." 
(411) 

Despite some disagreement in the literature, however, most other studies have not 
found significant mode effects in responses gathered electronically. Erdman, Klein and 
Greist (1983) found little difference between computer-administered and paper survey 
reports of drug use/abuse. Skinner and Allen (1983) found no significant difference 
between self-reported levels of alcohol, drug and tobacco use reported in face-to-face 
interviews versus those reported via computerized questionnaires. And in direct conflict 
with Kiesler and Sproull's results, the computer responses in their study indicated slightly 
lower reported frequencies of alcohol and marijuana use. 

Helgeson and Ursic (1989) evaluated decision process equivalency of 
undergraduate business students via electronic and paper data collection in part by 
comparing how the substance of answers changed when the order of survey questions 
was changed. They found no significant differences between the content of responses 
gathered electronically versus on paper; however, they found respondents decision 
processes to be more stable in the electronic medium. As the extremity of scale anchors 
changed, answers in the electronic medium remained more stable than those on paper. 
Booth-Kewley, Edwards and Rosenfeld (1992) surveyed male Navy recruits with 
Paulhus’s Balanced Inventory of Desirable Responding (1984), varying the medium in 
which they responded (electronic and paper) and the level of anonymity of the 
respondents. While they found a significant variance of response in relation to the level 
of anonymity, the effects of the survey medium were insignificant. 
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Synodinos, Papacostas and Okimoto (1994) administered a survey to randomly- 
and self-selected airport users via computer terminal and to randomly-selected users via 
personal interview. They found no significance differences in the responses between 
computer respondents and those personally interviewed, but, predictably, did find 
significant differences between self-selected and randomly-selected respondents. In their 
survey about Usenet newsgroup users’ attitudes toward Internet commercialization, 

Mehta and Sivadas (1995) found no difference between their email and regular mail 
responses. 

Bachmann, Elfrink and Vazzana (1996) surveyed business school deans and 
division chairpersons and found no significant difference between responses to the email 
and regular mail instrument. Bertot and McClure (1997) surveyed public libraries across 
the country via the \Veb about Internet use, and at the time of publication had found no 
response-rate bias on the basis of the population size of responding libraries’ legal service 
area or region (their study was ongoing). They did receive a greater response from some 
geographic regions — 34.8% from the Midwest and 28.2% from the West compared to 
19.6% from the Northeast and 17.4% from the South— but "it is unclear as to whether the 
percentages of electronic survey respondents by region and population of legal service 
area correspond to public library Internet connectivity in general by those strata." (174) 

A few studies did find differences between responses from electronic and paper 
instruments, but felt they were the result of other factors than the survey mode or that 
they were within acceptable limits. Miller, Daly, Wood, Brooks and Roper (1996) found 
a difference in response content between their email and paper surveys of professional 
computer scientists, but they attributed it to the two versions of the survey reaching 
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different audiences defined by job position and concluded that little self-selection bias 
(which can be interpreted as sampling frame bias) was evident m their study. Morphew 
and Williams (1998) determined email surveys to have a "sizable nsk of nonresponse 
bias due to low response rates" but that the risk "is on the order of that associated with 

postal surveys." (p. 52) 

A few studies did note apparent mode effects but they may actually have been 
sampling frame effects, specifically differences in demographics and computer 
experience. Shaw and Davis (1996) reported significant differences in responses between 
their electronic and paper groups in a survey of Modem Language Association members, 
but these corresponded to demographic differences between the respondents in the two 
groups, especially that electronic respondents were much more experienced with 
electronic technology. These participants were more likely to have a computer at home, 
more likely to use email, and more likely to use online library catalogs. Similarly, 
Anderson and Gansneder (1995) noted that comparisons of computer-monitored data 
between respondents and non-respondents indicated the former were more likely to use 
the computer and for longer periods of time. Their survey collected mainly demographic 
and electronic experience data, so substantive differences could not be measured. 

Finally, Kaufman, Carlozzi, Boswell, Barnes, Wheeler-Scruggs and Levy (1997) found in 
their survey of gays, bisexuals and lesbians about therapist selection that respondents to 
their electronic survey were more open about their sexual orientation than respondents to 
the paper instrument. The authors attributed the difference to the demographic 
differences between the two groups: "The email sample was younger, more educated and 
had higher reported incomes than the traditional sample...." (295-6) These demographic 
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differences correspond to the general demographics of Internet users, although the paper 
and electronic groups in this study were both balanced between men and women. In the 
other studies, the selection parameters for the samples were also questionable: the 
assumptions made by the researchers may have involved faulty logic. Shaw and Davis 
selected members of a professional organization as being electronically savvy because 
the organization they supported chose to support the development of an important 
electronic bibliographic database. Modem Languages Association International 
Bibliography. Likewise, both Anderson and Gansneder (1995) and Kaufman et al. 

(1997) recruited participants for their electronic instruments from listservs, assuming a 
uniformity of sample that may not have actually existed. 

The population of interest to the current study was academic librarians in North 

Carolina and was not chosen from a group organized on the basis of Internet 

participation. Thus, the first three hypotheses for this study were: 

Hypothesis 1: Respondents to the Web survey will not exhibit significantly 
different demographics from respondents to the paper survey. 

Hypothesis 2: Respondents to the Web survey will give responses that are not 
significantly different from responses to the paper survey. 

* 1 * Hypothesis 3: Respondents will not provide significantly different 
answers to the Web survey whether they were notified of it by email or 
postal mail. 

Advantages of Electronic Surveys 

Electronic data gathering has significant advantages which are agreed upon in the 
literature. (Most of the studies cited here used electronic mail to distnbute their surveys.) 
First, studies using electronic surveys note the low cost of administering them (Roselle 
and Neufeld, 1998; Berge and Collins, 1996; Clayton, Applebee and Pascoe, 1996; Miller 
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et al., 1996; Anderson and Gansneder, 1995; Kiesler and SprouII, 1986; Erdman et al, 
1983). Few state their actual costs, but electronic surveys inevitably eliminate the need to 
copy surveys, as well as the cost of postage, usually the major expense in postal surveys. 

Second, response is very fast. Several studies received the majority of their 
responses within one to two weeks of posting surveys (Roselle and Neufeld, 1998; 
Meehan and Bums, 1997; Berge and Collins,1996; Miller et al., 1996; Anderson and 
Gansneder, 1995; Mehta and Sivadas, 1995). Berge and Collins (1996) received their 
first response within twenty minutes of releasing their survey. Meehan and Bums (1997) 
received 39% of their responses within twenty-four hours. Mehta and Sivadas (1995) 
received more than half of all their responses within two to three days. Swoboda, 
Miihlberger, Weitkunat and Schneeweib (1997) received 90% of their 1,713 responses 
within 4 days. Morphew and Williams (1998) argue that multiple follow-up mailings 
make electronic survey periods comparable to mail and telephone survey periods, but 
they seem to be the only objectors. 

Good response rates are less uniformly agreed upon as a benefit of electronic 
surveys. Several email surveys have received response rates fully on par with traditional 
instruments. Roselle and Neufeld (1998) studied the effectiveness of email followup 
messages for a traditional postal survey. They received responses from 85.3% of the 
participants who received the email followup, compared to a 79.8% response rate from 
participants who received a postcard followup. Their overall response rate was 83%. 
Anderson and Gansneder (1995) achieved a 68% response rate to their email survey, 
excluding from their calculation a number of people who, according to computer data, 
did not read their email during the survey period. (Their response rate including those 
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people was 58%.) Walsh et al. (1992) received a 76% response rate to their email survey 
of 300 oceanographers. In addition, they received responses from an additional 104 
individuals spontaneously asking to participate. (The researchers analyzed this self- 
selected group separately from their original stratified random sample.) 

Other studies note lower response to email surveys than paper instruments used 
for the same surveys, but only slightly lower, as demonstrated by Table 1. 



Table 1. Comparison of Response Rates by Survey Medium 



Study 


Paper 

Response Rate 
(Percentage) 


Electronic 
Response Rate 
(Percentage) 


Bachmann et al. (1996) 


66 


53 


Shaw and Davis (1996) 


41 


37 


Kiesler and Sproull (1986) 


75 


67 


Sproull (1986) 


87 


73 


Miller et al. (1996) 


30 


*19 



*See paragraph below 



Only one survey reported an electronic response rate that was small enough as to 
be almost unusable. Miller et al. (1996) experienced significantly different response rates 
to their electronic versus their postal mail survey. The postal mail survey returned a 30% 
response rate. Although they could not conclusively state the size of the audience the 
electronic survey reached because they distributed the survey to a newsgroup, the authors 
based their estimated response rate on the average monthly postings per week of the 
newsgroup. Even using this very rough estimate of the number of recipients of the 
survey, the response rate for the email survey is 19%-and it could possibly be even lower 
than that if the number of recipients was underestimated. 
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Three other studies reported low response to their electronic surveys, but had not 
conducted more traditional surveys with which to compare them. Meehan and Bums 
(1997) reported electronic returns of approximately 23.6% from a survey of secondary 
school teachers and administrators. Smith (1997) reported a virtually unusable response 
rate to her electronic survey, but attributed this to technical difficulties respondents 
encountered with her instrument-some browser programs were unable to properly 
process respondents' completed surveys. Swoboda et al. (1997) received a 20% response 
rate to their survey about problems facing the world (political, social, etc.). It could be 
argued that in this case the low response is partially due to the target audience not being 
highly invested in the results of the questionnaire. Their survey was sent to 200 
randomly selected newsgroups focused on a variety of subjects, so the individuals it 
reached were not as concerned about participating as if they had been, say, international 
affairs analysts or environmentalists. 

The audience targeted by the current study is impacted daily in their professional 
work by the program which is the subject of the survey. Consequently, one would expect 
to have a high response rate regardless of the survey medium. The overall advantages 
reported for e-mailed surveys should also be present for a Web survey, so the next two 
hypotheses of the study were: 

❖ Hypothesis 4: The response rate of a Web-based survey will be no different 
from that of a paper survey. 

❖ Hypothesis 5; At least 50 percent of the total number of responses to the Web 
survey will be returned in one week. 
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Publicizing Electronic Surveys 

Researchers are often interested in special populations for their research, and seek 
efficient ways to contact large numbers of a particular group rapidly. Listservs and 
electronic bulletin boards represent “large populations [which] are well-defined in terms 
of a particular phenomenon." (1 13, Miller et al., 1996) As demonstrated in the previous 
section, studies are beginning to demonstrate the efficiency of electronic media for 
reaching particular audiences, especially ones that are geographically diverse. The main 
difference for surveys posted on the Internet from those distributed by electronic mail is 
targeting. 

Web surveys must be publicized. No audience will automatically see it without 
some promotional effort on the part of the researcher. Listservs and links on organization 
Web pages can be effective ways to advertise a survey; however, both strategies share the 
sampling bias of the Internet as noted earlier. In addition, they may be more or less 
effective depending on many factors such as user traffic, subject of the survey, and so 
forth. And these methods are not at all precise in targeting. A researcher cannot be 
completely sure of what population(s) they actually reach in these ways. Finally, they 
also make it impossible to calculate precise response rates. Membership figures for a 
listserv vary widely over any given period of time as individuals choose to withdraw 
from or join the list. Web page traffic may fluctuate widely depending on how often an 
organization's members seek updates or information. Also, an email sent to one listserv 
may be reposted to other groups. In both cases, the researcher cannot know the overall 
number of individuals who saw the advertisement, but only the number of people who 
respond (Berge and Collins, 1996; Miller et al., 1996; Walsh et al., 1992). 
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A way to circumvent this problem with en mass advertising is to use targeted 
email (Anderson and Gansneder, 1995; Shaw and Davis, 1995; Kiesler and Sproull, 

1986). Using personal email addresses is usually as specific as postal mailing. Many 
professional directories now include members' personal email addresses. Researchers 
can randomly choose participants from the directory in the same way that they might 
select a sample of mailing addresses for a mail survey. It is advisable to type each 
individual's address into a separate message to avoid compromising other participants 
privacy, rather than send one message to all participants. Functions such as “copy” and 
“paste” make this process fairly rapid. Anderson and Gansneder (1995) also note that 
addressing emails individually personalizes the appeal for response. 

Apart from the time involved, one would expect this method of advertising to be 
highly efficient and have the added benefit of enabling the researcher to calculate an 
exact response rate. Mailed notices are a more traditional means of providing 
preliminary notice of a survey, but respondents may be discouraged from responding by 
needing to take the extra step of going to the Web to answer the survey. The final 
hypothesis was: 

*> Hypothesis 6: Of the two methods of publicizing a Web survey, postal 
mail and email, email is the more efficient one. More responses will 
arrive more quickly from the group notified by email. 

Summary of Research Questions 

Due to the skew of user demographics, Internet populations are likely to provide a 
poor sample for research surveys. However, the sole fact of administenng a survey via 
the World Wide Web does not necessarily introduce such bias. Electronic surveys, 
whether distributed by email or the Web, are distinctly faster and less costly than 
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traditional postal surveys, but must be carefully publicized to produce response rates 
equivalent to more traditional methods. These observations have resulted in the 

following hypotheses for the current study; 

❖ Hypothesis 1: Respondents to the Web survey will not exhibit significantly 
different demographics from respondents to the paper survey. 

Hypothesis 2: Respondents to the Web survey will give responses that are not 
significantly different from responses to the paper survey. 

❖ Hypothesis 3; Respondents will not provide significantly different 
answers to the Web survey whether they were notified of it by email or 
postal mail. 

Hypothesis 4; The response rate of a Web-based survey will be no different 
from that of a paper survey. 

<♦ Hypothesis 5: At least 50 percent of the total number of responses to the 
Websed survey will be returned in one week. 

❖ Hypothesis 6: Of the two methods of publicizing a Web survey, postal 

mail and email, email is the more efficient one. More responses will 
arrive more quickly from the group notified by email. 



III. Methodology 

Academic librarians in North Carolina were surveyed about their attitudes toward 
NC LIVE, a state-wide digital library initiative. After selecting a stratified random 
sample of academic reference librarians in North Carolina for this study, the sample was 
divided into three groups; two to receive notice of the electronic survey, one to receive 
the paper survey. A software program was chosen to process the Web survey responses 
and forward the results by email to the researcher. Once the survey was administered, 
returned surveys were tracked for date of receipt. Data entry was completed in SPSS 9.0 
One-way analysis of variance and chi-square tests were utilized to analyze differences 
between survey groups. Bonferroni post-hoc analyses were used to further analyze 
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statistically significant ANOVAs. Spearman correlations were used to test relationships 
among ordinal data, and Pearson correlations were used to test relationships among 
interval data. 

Population 

The population for this study was academic reference librarians in North Carolina. 
A stratified random sample of 400 academic librarians was drawn from a combination of 
sources. The membership lists of the academic library sections of the North Carolina 
Library Association and the American Library Association provided 275 individuals for 
the survey, and another 125 were researched from institutions' Web page staff directories. 
The stratification method seems unlikely to have produced significant bias since email 
addresses were relatively easy to locate for all three strata in the sample. Difficulty 
seemed to arise with specific institutions rather than any given classification of institution 
(e.g., community college versus university). Representatives were included from across 
the state in all types of academic libraries, from large university libraries to community 
colleges and private colleges. Large numbers of individuals from the larger staffs in 
university libraries were offset by the greater number of community and small colleges. 

The total group of 400 was randomly divided into four groups: one received the 
paper survey, one received the paper announcement of the Web survey and a third 
received the email announcement of the Web survey. The fourth group was randomly 
divided into three further groups to provide substitutions for the first three groups. Such 
substitutions occurred when an individual's title made it clear he or she was unlikely to 

1 The exact numbers of representatives from universities, community colleges, etc., were not calculated 
because those affiliations were not always apparent from the associations’ mailing lists. 
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work on the reference desk, or if their mailing address or email address was incomplete, 
erroneous or unavailable. Overall, 37 substitutions occurred, 22 of them in the group 
receiving the paper survey and most often because of inappropriate position title. 

For the group receiving the email notice, it was necessary to research individuals’ 
work email addresses because no listserv exclusively serves academic librarians in North 
Carolina, and the membership lists mentioned above included only regular postal mail 
addresses. This research entailed approximately fifteen hours of work. Actually 
emailing the survey notices required approximately one and a half hours, both for the first 
and second notices. 




The Survey 

2 

This study sought to survey a population experienced with electronic resources, 
by electronic and paper surveys, about their attitudes toward NC LIVE, a new program to 
provide North Carolina academic and public libraries with collective access to a wide 
variety of electronic databases.^ "NC LIVE is a statewide electronic library project of the 
libraries of North Carolina designed to strengthen the delivery of information statewide to 
enhance education, economic development, and the overall quality of life." (State of 
North Carolina, 1998) This program provides access to over 3,500 general magazines. 
Journals and newspapers via approximately forty licensed databases, including several 
full text vendors such as ProQuest and EBSCOhost. The resources included cover a wide 
range of disciplines, from religion to politics, from psychology to recreation, and include 
general reference resources as well as subject-specific ones. 

2 "Electronic resources" includes computers, electronic mail, electronic databases and other software. 

3 Summary results of this survey are presented in Appendix E, since they are not the primary focus of this 
Study. 
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The State Library, with its partners, introduced North Carolina Libraries for 
Virtual Education (NC LIVE) in the spring of 1998. Partners include public libraries and 
community college, private college and university libraries around the state. Although 
larger libraries already had access to some of the resources the partners decided to offer 
through the program, the cost benefit of consortially negotiated licenses made it attractive 
to join. A structured introductory program provided optional training to librarians across 
the state before and during the NC LIVE premier, and continues to provide workshops as 
needed. The paper survey is presented as Appendix A, and the printed version of the 
Web survey as Appendix B. The cover letter for the paper survey as well as the paper 
and email announcements of the Web survey appear in Appendix C. 

Procedures for Distributing the Surveys 

The paper survey and mail notice of the Web survey were sent several days ahead 
of the email notice of the Web survey in an effort to ensure that all instruments arrived at 
approximately the same time. Recipients of the paper survey were invited to complete 
the questionnaire and return it within one month, and received a follow up notice two 
weeks after the original mailing. Recipients of the paper and email notices of the Web 
survey were invited to complete the questionnaire within two weeks, and received a 
follow up notice after one week. 

The Web survey was as nearly a duplicate of the paper instrument as possible. It 
was created using a combination of FrontPage 98, an HTML editor program, and direct 
HTML programming. The form for the Web survey was created with Gform, a program 
which relays a respondent’s answers to the server on which the survey is mounted. The 
server, in turn, encodes the information as an email message to the address specified by 

O 
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the researcher, including no information about the respondent. This ensured responses 
would be, not just confidential, but anonymous. Browsers do collect information about 
users as they respond to Web surveys, including their IP address and host domain (the 
specific address of the computer they use and the general address of the host, such as 
“.unc.edu”). It would be possible to collect this information and identify respondents’ 
institutions if they use their work computer to respond, but it would be nearly impossible 
to discover the individual user (Dixon, 1999). Anonymity is complete, although this does 
present problems for any subsequent follow up. (If participants had been invited to 
include their email address voluntarily along with their responses, follow up would have 
been possible.) 

Gform assisted in differentiation and coding, as well as anonymity. The two Web 
survey groups (paper notice and email notice) were directed to two separate but identical 
Web pages. The program enabled the researcher to insert identifying subject lines in the 
server’s email indicating from which Web page the response had been submitted, clearly 
delineating the responses of the two groups. In addition, Gform will convey to the server 
whatever information a programmer associates with each answer, enabling coding to be 
assigned to each answer at the time the Web form is constructed. Although more 
advanced software is available which can deliver response information directly into a 
database file, working with Gform on this more basic level helped speed the manual data 

entry without the high cost of such software. 

One significant difference between the electronic and paper versions of the survey 
concerned information about the respondent’s library’s Carnegie classification. In 
preliminary testing, many respondents were unsure of their library’s category. In an 
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effort to boost response to this item, the Web survey linked to a Web site presenting a list 
of institutions in each category. This had significant unforeseen ramifications due to 
inadequate pre-testing. Shortly after the email announcement survey was sent, a 
respondent notified the researcher that following this link cleared all previously marked 
responses on the Web form. Since the classification question occurred at the end of the 
quest, onnatre, answers to v.rtually all questions were lost and ,t was annoying to have to 
do the enfire survey again. A wam.ng was tmmediately tnserted about the problem on 
both survey Web pages, as well as a suggestion to open a separate browser window to 
follow the link; but several people had already responded without noticing and several 
responded subsequently with blank forms. The consequences of this problem are 

discussed further in the Results section. 

Another problem which manifested itself in the data analysis concerned the 

question about respondents' primary work responsibtl.ties. The origtnal quest, on .nvited 
respondents to note whether their primary work was technical, public servtce or 
managerial. In the Web survey the options were controlled by radio buttons, enabling a 
respondent to select only one answer. In the paper survey this preference for one answer 
wasn't expressed (e.g, “Select only one''), so a number of respondents marked more than 
one response. Handling of this problem will be discussed further in the Results section. 

IV. Results 

A total of 130 people responded to the survey overall, a response rate of 43.33%. 
53 respondents had received the paper survey; 33 had received the mail announcement of 
the Web survey; and 44 had received the email announcement of the Web survey. Of the 
53 paper responses, 5 1 were usable; of the 33 mail announcement responses, 27 were 
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usable; and of the 44 email announcement responses, 39 were usable. This results in an 
overall usable return rate of 39.33%. On the basis of these 118 usable responses, the 
usable paper survey response rate was 43.22%, the mail announcement rate 22.88%, and 
the email announcement rate 33.05%. 

Hypothesis 1: Respondents to the Web survey will not exhibit significantly 
different demographics from respondents to the paper survey. 

Hypothesis 1 was not rejected. Tests of the three survey groups on variables 

relating to respondents' demographic characteristics found no significant differences (at a 

.05 level of significance) between respondents to the paper versus the Web survey. 

Demographic characteristics tested include age, sex, library education and amount of 

time respondents have worked in libraries among other things. Details of the tests are 

presented in Appendix 4. 

Hypothesis 2: Respondents to the Web survey will give responses that are not 
significantly different from responses to the paper survey; and 

Hypothesis 3; Respondents will not respond significantly differently to 
the Web survey whether they were notified of it by email or postal mail. 

Hypothesis 2 was not rejected. The three survey groups were compared on the 

basis of variables relating to respondents' opinions about the NC LIVE program and those 

relating to respondents’ computer experience. Opinion variables tested include five 

positive and four negative aspects of the NC LIVE program. Computer experience 

variables tested include questions about respondents’ frequency of use of various kinds of 

electronic resources and home access to computers. The tests found no significant 

differences (at a .05 level of significance) between respondents to the paper versus the 

Web survey, except for one variable. An ANOVA demonstrated a significant 

relationship between the mean demand for computers prior to the start of the NC LIVE 
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program for responses from the three survey groups (F=37.769 with 2 df, p=0.000). The 
Bonferroni post-hoc analysis indicated a significant relationship only between the paper 
survey and the Web survey group notified by email (see Appendix D for more details). 

Details of the tests are presented in Appendix D. 

On the basis of these same tests of variables relating to respondents' opinions 
about the NC LIVE program and those relating to respondents’ computer experience. 
Hypothesis 3 was not rejected. The tests found no significant differences (at a .05 level 
of significance) among respondents to the Web survey, whether they were notified of it 
by mail or email. 

One variable that at first appeared to show a difference between the Web and 
paper instruments was found to be not significant upon further analysis. In the data entry 
stage, with the goal of capturing as much information as possible, a fourth category was 
noted for the question about primary work, “combination,” to make note of those paper 
surveys where the respondent had marked more than one category. In the preliminary 
data analysis stage, the fact that this fourth category was artificially weighted toward the 
paper survey was forgotten. An ANOVA seemed to indicate that those who marked 
“managerial” as their primary work were more likely to answer the paper survey. When 
the mistake was realized, the responses in this category were reclassified into the work 
category with the largest number of responses, forcing a somewhat artificial designation. 
So, for example, if someone marked both “technical” and managerial categories, the 
response was reclassified in the “managerial” set because there were more responses in 
that group than in the “technical” set. In re-running the significance tests, no 
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relationship was indicated between respondents’ primary work category and their 

likelihood of answering via the Web or on paper. 

•> Hypothesis 4: The response rate of a Web survey will be no different from that of 
a paper survey. 

Hypothesis 4 was rejected. The Web survey in this study did not have the same 
response rate as the paper survey. The paper survey achieved a response rate ten 
percentage points higher than the Web survey group notified by email and more than 
twenty percentage points higher than the Web survey group notified by mail. Certainly, 
to achieve a comparable response rate to a paper instrument, it is critical that the 
electronic survey be free of technical problems. The effects of the problem link from the 
Carnegie classification item in the Web survey were significant, invalidating 15.15% of 
the total response to the Web survey. Yet even if all responses to the Web survey had 
been valid, the Web survey’s response rate would not have matched that of the paper 
survey. Consideration of the literature review seems to confirm this as a general trend for 
electronic surveys in comparison with paper surveys, whether the electronic survey is 
administered via email or the Web. Electronic surveys often seem to generate lower 
response rates than paper surveys, although the degree of difference between the two 
rates may vary according to how well each survey is promoted, what followup and 
motivational procedures are employed, and the general responsiveness of the population 

surveyed. 

Hypothesis 5: At least 50 percent of the total number of responses to the Web 
survey will be returned in one week. 

Hypothesis 5 was rejected. Fifty percent of responses to the Web survey were not 
received within one week. Only 27.5% of the total number of responses to the Web 
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survey was received within the first week. An ANOVA showed a clear relationship 
between the date the returned survey was received and which survey the respondent 
completed (F=37.769 with 2 df, p=0.000). Furthermore, a Bonferroni post-hoc analysis 
indicated a strong difference between the mean of the paper survey and those of both the 
Web survey groups (p=.000 for each comparison); however, there was no significant 
difference between the two Web survey groups (see Appendix D for more details). 

Certainly, responses from the email announcement group were returned the fastest 
of the three groups. The first response was returned within an hour and a half of sending 
the announcement. Of the 52 responses received in the first seven days, 27 (65.85%) 
were from this group. Fourteen were from the Web survey/mail notice group, and eleven 
from the paper survey group (see Figure 1, next page). 

The electronic returns did not have quite as large a lead over the paper returns as 
expected, however: while 30% of responses received from the Web survey notified by 
email group arrived by the seventh day from mailing, only 14% of the total from the Web 
survey notified by mail group were received by that time, compared to 21.57% of the 
total responses received from the paper survey group. (Only usable responses were 
counted for these calculations.) This may be in part because the paper surveys were 
mailed earlier in hopes that the paper and electronic instruments would be received at the 
same time. 
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Figure 1. Date Received (by Survey Group) 




Paper survey 


Web Survey/Notified by Email 


Web Survey/Notified by Mail 





<♦ Hypothesis 6: Of the two methods of publicizing a Web survey, postal 
mail and email, email is the more efficient one. 

Hypothesis 6 was not rejected. The email notice of the Web survey was more 
efficient in eliciting responses than the mailed notice. An ANOVA demonstrated a 
significant relationship between the mean date received for responses from the three 
survey groups (F=37.769 with 2 df, p=0.000). However, the Bonferroni post-hoc 
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analysis indicated significant relationships only between the paper survey and each of the 
Web groups, not between the two Web groups (see Appendix D for more details). 

V. Discussion 

Comparisons of the demographics of the respondents to the Web and paper 
surveys found no significant differences between the demographic make up of the three 
survey groups. How old a person was, nor their sex, nor what degree of education they 
had attained influenced which survey they answered. Details of the tests are available in 

Appendix 4. 

Comparisons of the responses to the Web and paper surveys found no significant 
differences between the content of responses from the three survey groups, except for one 
variable. Since no other variables showed significant differences between the responses 
to the paper and Web surveys, and since there was no difference between the mean 
demand for computers prior to the start of the NC LIVE program between the paper 
survey and the Web survey group notified by mail, this result is puzzling. Overall, 
attitude toward the NC LIVE program was not affected by the medium in which 
respondents answered, nor was there significant difference between the groups in terms 
of experience with electronic resources. Details of these tests and some further 

discussion are available in Appendix D. 

The overall response rate was higher for the paper survey than for either of the 
Web survey groups (43.22% for the paper survey versus 33.05% for the Web survey 
notified by email and 22.88% for the Web survey notified by regular mail). When only 
the most thorough method will do and a high response rate is critical, paper and pencil 
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still hold the lead over electronic means as a survey method. Considerations for future 
researchers will include time constraints, cost, and the motivation of participants to 
respond. Significantly more responses were received to the paper survey than to the Web 
survey. Paper surveys remain a more productive medium for response, even among an 
electronically proficient community. 

Response from the Web survey groups was faster overall than from the paper 
survey group, but not by as wide a margin as originally expected. Counting only usable 
responses, 30% of responses received from the Web survey notified by email group 
arrived by day seven, but only 14% from the Web survey notified by mail group, 
compared to 21.57% of the total responses received from the paper survey group which 
arrived by day seven. Promotion of the Web survey was clearly accomplished more 
efficiently via email than mail. Even before discounting unusable responses, the email 
group filling out the Web survey was more likely to respond than the mail group. It 
seems likely that the better early response to the Web survey is partly due to its earlier 
deadline. The cover letter/email made clear that responses were requested by May 1, 
and respondents clearly made an effort to comply with that. It would be interesting to 
compare long-term response with identical deadlines. 

The most significant difference between the two instruments from a data 
standpoint was the flexibility the paper instrument provided respondents in how they 
answered. Respondents felt free to make comments about questions they didn t 
understand or felt were ambiguous. They often provided different answers to the same 
question to illustrate the different ways of interpreting it. The Web survey forced 
respondents to answer in particular ways, with no easy means for providing comments. 
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Confusion on the part of the researcher over the significance of respondents’ primary 
work category arose because respondents to the paper survey were able to mark more 
than one answer to the question, whereas the web survey respondents were permitted 
only one choice by the radio button answer selector. Choice of radio buttons or check 
boxes gives researchers a greater amount of control over how they want respondents to 
answer questions, but it also results in receipt of less information overall from the 
respondents. It might have been very important for the study to know that some 
academic librarians have combinations of different kinds of work in their jobs. If only 
the Web survey had been administered, this fact would not have come to light at all. 

For this population, the use of Web surveys seems a reasonable alternative to 
postal surveys or telephone interviews, depending on the research question. Clearly, the 
response rate in this study is rather low. Dillman (1978) notes that a response rate such 
as that achieved for the Web survey group notified by email— 33%— leaves the majority 
of the population unsurveyed. If the survey in some way discouraged from responding 
people who all felt the same way, the majority opinion would remain unrepresented by 
these results. Given that no significant difference was found between survey groups on 
the basis of the demographic variables (see Appendix D), it is unlikely that such glaring 
bias would exist among the non-respondents. It is, however, a possibility. Further 
research into what motivations can be used effectively with electronic surveys would be 

useful. 

Technical Difficulties 

Perhaps the greatest caveat this study offers for future Web surveys is to caution 
that survey authors find expert consultation on the technical aspects of the survey and 
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conduct fully as thorough a pre-test on an electronic survey as on a paper instrument, 
even if they include exactly the same items. Smith (1997) provided the only comparison 
of email and Web surveys found in this study, but she encountered serious technical 
difficulties in publishing her Web survey to the extent that she was unable to gather 
sufficient data to make a meaningful comparison between the two modes. Her survey 
format was incompatible with at least two types of Internet browsers, preventing 
respondents from submitting their completed surveys. Due to inadequate pre-testing of 
the electronic instrument for the current study, a significant number of responses to the 
Web survey were lost. Researchers should be very sure they have anticipated and 
diagnosed as many technical issues as possible. A few other possible problems, among 
many, include the way in which the questions are displayed by different browsers, 
whether different browsers can interpret the form protocol used, and problems with early 
generation computers interpreting advanced applications, such as Javascript and frames. 



VI. Conclusion 

The advantages the Web survey holds in administration time and cost almost 
force researchers to consider it as a serious alternative to more traditional survey media, 
Miller et al. (1996) suggested the efficiency of electronic data collection method 
justified its use, particularly for exploratory research and for populations that have no 
sampling frame. However, I would disagree. The greater freedom paper instruments 
offer respondents should be an important consideration, particularly for exploratory 
research. Of course researchers must weigh many factors in choosing an appropriate 
survey medium, but when cost and time constraints outweigh other considerations, the 
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Web may be an adequate medium if sufficient comment areas are supplied within the 
survey form and the sampling frame for the population is not biased. 
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Survey of Academic Librarians’ Experience with NC LIVE 



PLEASE RETURN BY MAY 15 . 

Your submission of the completed questionnaire will be taken as an indication of your consent to participate. 



Section T - NC LIVE^ 



1 . 

2 . 



If you are currently employed in an academic library, does it have NC LIVE access? 



Yes □ No □ 



a. If you answered yes to question 1 , how long has your library had NC LIVE? 



months 



b. If you answered no to question 1, do you know when you will get access to it? 
to Section II, cit the bottom of the next pcige. 



(month, year) 



—Please proceed 



3. 



In your view, what are the greatest benefits of having NC LIVE? 



Please check all that apply. 



□ Helps staff learn more electronic resources 

□ Encouragement of patrons to explore electronic resources 

□ Provides a marketing tool to promote the library to the community 



□ Greater variety of resources for patrons 

□ Brings more people into the library 

□ Other: 



4. 



In your view 



lew, what are the greatest detriments of having NC LIVE? Please check all that apply. 



j A ^ cfnff Q Tnrreases natron reliance on electronic resources even 

□ Additional bibliographic inslrnction demands on staff □ „„„ appropriate 

□ Heavy demand for computers ^ 

□ Strains staff coverage at the reference desk Cl Other; 



5. How many public computer stations in the library where you work give patrons access to NC LIVE . 

6. Can patrons access multiple software applications on these computer stations, including software like word-processing 

programs? Yes □ No □ 



7. If yes, please check software applications available. Please check all that apply. 

n Wadsheet nromm □ Internet □ Library OP AC 

□ Word-processing Cl Spreaasneei program 

□ Scanning programs (image or text) □ Other; 



Assess the level of demand for the library’s public computer stations 
1 2 3 

less than 5 users per hour 
at peak periods 



before NC LIVE was available (1 is least; 5 is greatest). 



more than 20 users per hour 
at peak periods 



^NC LIVE Mission statement; “NC LIVE (North Carolina Libraries for^^^^^^^^^ 

the libraries of North Carolina designed to strengthen t e e ivery o i ,pnresented bv the State Library of North Carolina, The 

development, and the overall quality of life. Working toget ^ ^ Sv^stem and the North Carolina Center for Independent Higher 

University of North Carolina, The North Carolina Community College System and North^Caro 

Education will collaborate to give all North Carolmians-students, facu y- Hbraries statewide ” For more information, see 

access to a range of electronic information resources and to the X^^?‘NC LIVE 

the State Library web site at http;//statelibrary.dcr.state.nc.us/ and select NC LIVE from the guicK 



1 



~5ip.ctif>n I - NC LIVE (continued) 



9. 



Assess 



the level of demand for these stations now with NC LIVE (I is least; 5 is greatest). 



10 . 



11 . 



12 . 



1 



more than 20 users per hour 
al peak periods 



less than 5 users per hour 
at peak periods 

Assess your own frequency of use of electronic indexes either on CD-ROM or via the Internet (^g., ERIC, Psychlit, 
MEDLINE) per hour at the reference desk prior to gaining NC LIVE (1 is least; 5 is greatest). Do not me u e uses 

library’s online catalog. 

2 3 4 ^ 

I more than 1 0 uses per hour 

less than 2 uses per hour 

Assess your own frequency of use of electronic indexes through NC LIVE per hour at the reference desk ( I is least; 5 is 
greatest). 



1 ^ 

less than 2 uses per hour 

Which resources within NC LIVE do you refer patrons to most frequently? 



more than 1 0 uses per hour 



13. How many databases in NC LIVE do you personally use regularly? 

14, Which ones do you use most frequently? 



15. 



16. 



Has NC LIVE increased the amount of bibliographic instruction you provide to patrons? 
NoQ YesQ 

If yes, how? Please check all that apply. □ time spent with individual patrons 

□Other: - 



□ number of group classes taught 



How much in-class training have you had in using NC LIVE (in hours)? 
Do you feel these workshops have provided sufficient training? 



Section II - Riectronic Library Ser vices/Experience 

Please indicate whether the library where you work offers patrons the following kinds of access 





Off 

Yes 


er? 

No 


1. 


Graphics-based World Wide Web searchmg/browsmg {e.g., using 
Netscape, Mosaic or Explorer software) 






2. 


Text-based World Wide Web browsing (e.g., using Lynx software) 






3. 


Email accounts 






4. 


Newsgroup services (e.g., access to newsgroup readers) 







O 
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n - El^^tronic Library .Services/Experience (continued} 





Off 

Yes 


sr? 

No 


5. Library's catalog online 






6. Commuting software {e.g., word-process, ng or spreadsheet 

programs) 






7. Other - Please specify: 







have referred to electronic literature databases, either on CD- 



8 Which unit below best describes the frequency with which you I 

ROM or via the Internet, during reference desk work in the last y • 

Every week □ Every month □ 



Every hour □ 



Every day □ 



Which unit below best describes the frequency with which you currently use such databases during reference desk work? 



Every hour □ 



Every day □ 



Every week □ 



Every month Q 



Please estimate the number of times you use each 
in a typical week: 



of the following electronic resources at work but outside of reference desk service 



la World Wide Web access {e.g., research or 

surfing) 

11. Electronic mail 



1 3. Searching or maintaining your library's online 



catalog 

"74^ Computing software (e.g., word-processing or 



spreadsheet programs) 

15. Other - Please specify : 



Never Less than 10 1 1 -25 26-50 




More than 50 



■Section m - Demographi c Information 

1. Age: 24 or below □ 25 

2. Sex: MaleQ Female □ 

3. Is English your first language? Yes □ 



35-44 □ 45-54 □ 55-64 □ 

If no, what is your first language? 



O 

ERIC 



65 or above □ 



39 



^ Section UT - Demographic Information (continued) 

4. Please check your highest level of library education completed: 
Associate’s degree □ Bachelor’s degree □ 

Please list any other non-library degrees you hold. 



5. 

6 . 

7. 

8 . 

9. 

10 . 

11 . 

12 . 

13. 



Paraprofessional library training Q 
M.L.S. □ Ph.D. □ 



Yes □ 



No □ 



Are you currently employed more than 30 hours in a academic library? 

If so, how long have you worked in that library? Years Months 

Overall, how long have you worked full-time in libraries (this one plus any other libraries where you may have been 
employed full-time)? Years Months 



What is your job title? 

How many hours each week do you work at the reference desk? 

SorlessQ 6-10 □ H-15 □ 16-20 □ 21-30 □ 30 or more □ 

Would you categorize your work as primarily public service (reference) □, primarily technical □. or primarily managerial □? 

If you answered "public service" above, do you mainly provide service to □ students □ faculty or □ members of the 
larger community? 



If you are currently working in an academic library, what is its Carnegie classification? 



□ Research University I 

□ Doctoral University II 



□ Research University II 

□ Master's (Comprehensive) 
University/College I 

□ Baccalaureate (Liberal Arts) College I □ Baccalaureate (Liberal Arts) College II 

□ Professional School/Specialized 
Institution 



□ Doctoral University I 

□ Master's (Comprehensive) 
University/College II 

□ Associate of Arts College 



14. 


If you are currently working in an academic library, is it. 


□ the main campus library or 

□ a departmental library (health sciences, law, art, etc.)? 


15. 


If you are currently working in a academic library, what is the size of the library s collection? volumes 


16. 


Do you have your own computer at home? 


YesQ 


NoQ 




17. 


If yes, do you use it more than once a week? 


YesQ 


No □ 




18. 


If you have a computer at home, do you have email 
access of any sort at home? 


YesQ 


No □ 




19. 


If yes, do you use it more than once a week? 


YesQ 


No □ 




20. 


Do you have Internet access at home? 


YesQ 


No □ 




21. 


If yes, do you use it more than once a week? 


YesQ 

YesQ 


No □ 
No □ 




22. 


Do you have your own computer at work? 


Please return this survey to: 




-End of the Survey- 






M. Matz 




Tliank you for Participating! 






602 Airport Rd., Apt. 205 
Chapel Hill, NC 27514 
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Survey of Academic Librarians’ Experience 



with NC LIVE 



Obligatory Consent Notice 



• This survey is part of a SILS master's paper project meant to glean information 
about the use of NCLIVE in academic libraries. Your help in filling out this 
survey will provide valuable information about the impact on academic library 
staffing of the public's increased access to electronic resources, as well as about 
aspects of electronic research. This survey has been announced to reference 
librarians in academic libraries across North Carolina. 

• This survey is completely voluntary and confidential. Responses are sent to my 
personal email address. No identifying numbers, names, or other personal 
information will be recorded in any fashion. 

• You have the right not to answer any question(s) that you prefer not to answer. 

• Questions regarding the research may be directed to Michele Matz 
(matzm@ils.unc.edu) . 

• Questions regarding your rights as participants in the research may be directed to: 

Academic Affairs Institutional Review Board 

David A. Eckerman, Chair 

CB#4100, 201 Bynum Hall 

The University of North Carolina at Chapel Hill 

Chapel Hill, North Carolina 27599-4100 

(919) 962-7761 

email: aa-irb-chair@unc.edu 



ittp://www. ils.unc.edu/matz/NCLIVE/index.html 
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• If you would like to request the results of this study, email Michele Matz 
(matzm@ils.unc.edu') . This email will not be associated with your survey 
response. 



Proceed to the Survey 
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Survey of Academic Librarians’ Experience 

with NC LIVE 

PLEASE COMPLETE BY MAY 1 



Section I - NC LIVE 

1. If you are currently employed in an academic library, does it have NC LIVE access? 
Yes No 



2. a. If you answered yes to question 1, how long has your library had NC LIVE? 

Cl 



months 



b. If you answered no to question 1, do you know when you will get access to it? 



(month, year) 



—Please proceed to Section II . 



3. In your view, what are the greatest benefits of having NC LIVE? Please check all 
that apply. 

Helps staff learn more electronic resources 
Greater variety of resources for patrons 
Encouragement of patrons to explore electronic resources 
Brings more people into the library 

Provides a marketing tool to promote the library to the community 
D Other; 



O 
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4. In your view, what are the greatest detriments of having NC LIVE? Please check all 
that apply. 

Additional bibliographic instruction demands on staff 

Increases patron reliance on electronic resources even when print 
resources are more appropriate 

Heavy demand for computers 



O 



Strains staff coverage at the reference desk 



D Other: 



5. How many publi c com puter stations in the library where you work give patrons 
access to NC LIVE? 



6. Can patrons access multiple software applications on these computer 
stations, including software like word-processing programs? 

Yes No 

7. If yes, please check software applications available. Please check all that apply. 

Word-processing ^ Spreadsheet program 

Library OPAC Scanning programs (image or text) 

n Internet C Other: f ' ' " 

8. Assess the level of demand for the library’s public computer stations before NC 
LIVE was available (1 is least; 5 is greatest). 

Cl 02 03 C>4 O5 
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(less than 5 users per hour (more than 20 users per h 

at peak periods) at peak peric 



9. Assess the level of demand for these stations now with NC LIVE (I is least; 5 is 
greatest). 

Cl 02 O3 O4 O5 



(less than 5 users per hour (more than 20 

at peak periods) a 



10. Assess your own frequency of use of electronic indexes either on CD-ROM or via 
the Internet (e.g., ERIC, Psychlit, MEDLINE) per hour at the reference desk prior to 
gaining NC LIVE (1 is least; 5 is greatest). Do not include uses of the library’s online 
catalog. 

Oi 02 O3 O4 O5 



(less than 2 uses per hour 



(more than H 



II. Assess your own frequency of use of electronic indexes through NC LIVE per hour 
at the reference desk (I is least; 5 is greatest). 

Oi 02 O3 O4 O5 



(less than 2 uses per hour 



(more than H 



12. Which resources within NC LIVE do you refer patrons to most frequently? 



13. How many databases in NC LIVE do you personally use regularly? 



O 

ERIC 



http://www.ils.unc.edu/matz/NCLIVE/survey.html 



45 



01/22/2000 



NC LIVE Survey 



Page 4 of 9 



14. Which ones do you use most frequently? 



15. Has NC LIVE increased the amount of bibliographic instruction you 
provide to patrons? No Yes ^ 



16. If yes, how? Please check all that apply. 

^ time spent with individual patrons D number of group classes taugh 

Other: 



17. How much in-class training have you had in using NC LIVE (in hours)? 

iZZZIj hours 

18. Do you feel these workshops have provided sufficient training? 

Yes No 



Section II - Electronic Library Services/Experience 

Please indicate whether the library where you work offers patrons the following kinds of 
access: 





Offer? 




Yes 


N( 


1. Graphics-based World Wide Web searching/browsing {e.g., using 
Netscape, Mosaic or Explorer software) 


O 


G 


2. Text-based World Wide Web browsing {e.g., using Lynx software) 


O 


o 


3. Email accounts 


C 


O 
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4. Newsgroup services (e.g., access to newsgroup readers) 


c 


c 


5. Library's catalog online 


O 


c 


6. Computing software {e.g., word-processing or spreadsheet programs) 


o 


o 


7. Other - Please specify: .. . 


c 


c 



8. Which unit below best describes the frequency with which you have referred to 
electronic literature databases, either on CD-ROM or via the Internet, during reference 
desk work in the last year? 



Every hour 



G 



Every day 



Every week 



G 



Every month ^ 



9. Which unit below best describes the frequency with which you currently use such 
databases during reference desk work? 

Every hour ^ Every day Every week Every month 



Please estimate the number of times you use each of the following electronic resources at 
work but outside of reference desk service in a typical week: 





Times Per Week 


Never 


Less 
than 10 


11-25 


26-50 


More 
than 50 


10. World Wide Web access (e.g., 

research or surfing) 


o 


G 


G 


G 


0 


1 1 . Electronic mail 


o 


O 


o 


o 


G 


12. Listservs/Newsgroups 


o 


G 


G 


G 


o 


13. Searching or maintaining your 
library's online catalog 


G 


Q 


G 


o 


o 


14. Computing software (e.g., word- 
processing or spreadsheet programs) 


G 


o 


C 


o 


G 


15. Other - Please specify: 


O 


o 


G 


G 


G 







O 
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Section III - Demo g ra ph ic Information 

1. What is your age? 



24 or below 



C 25-34 



45-54 



55-64 



35-44 

65 or above 



2. What is your gender? Male ^ Female 

3. Is English your first language? Yes No 

If no, what is your first language? ...I 

4. Please check your highest level of library education completed: 

Paraprofessional library training ^ Associate’s degree 

Bachelor’s degree ^ M.L.S. 

^ Ph.D. 



5. Please list any other non-library graduate degrees you hold. 



6. Are you currently employed more than 30 hours in an academic library? 
Yes No 



er|c 
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7. If so, how long have you worked in that library? 



Years I J Months 



8. Overall, how long have you worked full-time in libraries (this one plus any other 
libraries where you may have been employed full-time)? 



Years 



Months i 



9. What is your job title? 




10. How many hours each week do you work at the reference desk? 
5 or less ^’6-10 ^’11-15 



G 16-20 



G 21-30 



30 or more 



11. Would you categorize your work as: 

^ primarily public service (reference), 
^ primarily technical, or 
^ primarily managerial ? 



12. If you answered "public service" above, do you mainly provide service to 



^ students. 



O 



faculty or 



members of the larger community? 



ERIC 
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13. If you are currently working in an academic library, what is its Carnegie 
classification ? {Part 1 on this page lists institutions in each category but following this 
link will clear your form. You might want to open a separate browser frame to check it.) 



Research University I 


Research University II 


Doctoral University I 


Doctoral University II 


Master's (Comprehensive) 


Master's (Comprehensi 


University/College I 


University/College II 


Baccalaureate (Liberal Arts) 
College I 


^ Baccalaureate (Liberal Arts) 
College II 


^ Associate of Arts Colle 


Professional School/Specialized Institution 





14. If you are currently working in an academic library, is it 

the main campus library or 

a departmental library (health sciences, law, art, etc.)? 

15. If you are currently working in an academic library, what is the size of the library’s 
collection? 



volumes 



16. Do you have your own computer at home? 



17. If yes, do you use it more than once a week? 



18. If you have a computer at home, do you have email access of any sort at 
home? 



19. If yes, do you use it more than once a week? 



Yes No 
Yes No 



Yes No 



Yes ^ No ^ 



O 
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20. Do you have Internet access at home? 



21. If yes, do you use it more than once a week? 



22. Do you have your own computer at work? 



Yes No 
Yes No 
Yes ^ No 



23. • Did you complete this survey 
at work, 



^ at home or 



^ at another location? 



24. How did you hear about 

Mail announcement 
From a colleague 



survey? 

Email announcement 
Other (Please specify): 



this 



End of the Survey 



If you would like to review your answers before submitting them, please return to the top 
of the survey now to do so, prior to clicking the submission button below. 

£ I „ Spbmyt iYOiLij^.cpm^ :?urv,ei|,, nowj, | : 



[ i^.. Clga|£:'EopTi. I 



NOTE: This button clears the form completely to let you start again. 



Thank you for your time! 



BEST COPY AVAILABLE 



o 
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20. Do you have Internet access at home? 



21. If yes, do you use it more than once a week? 



22. Do you have your own computer at work? 



23. Did you complete this survey 

at work, ^ at home or 



Yes No ^ 
Yes ^ No 
Yes ^ No 



at another location? 



24. How did you hear about this survey? 



Mail announcement 
From a colleague 



^ Email announcement 
Other (Please specify): 



End of the Survey 

If you would like to review your answers before submitting them, please return to the top 
of the survey now to do so, prior to clicking the submission button below. 

E3I1I ! ' If lil 



NOTE: This button clears the form completely to let you start again. 



Thank you for your time! 



BEST COPY AVAILABLE 



o 
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THE UNIVERSITY OF NORTH CAROLINA 
AT 

CHAPEL HILL 



The University of North Carolina at Chapel Hill 
CB# 3360, 100 Manning Hall 
Chapel Hill, N.C. 27599-3360 

April 19, 1999 



School of Information and Library Science 
Phone# (9 19) 962-8366 
Fax# (919) 962-8071 



Dear Academic Librarian: 

I am conducting a survey for my master’s thesis of the impact NC LIVE has on patrons’ use of electronic 
services in academic libraries. The objective of this research project is to understand the impact on academic 
library staffing of the public’s increased access to electronic resources, as well as to study aspects of 
electronic research. Understanding the impact of programs like NC LIVE is important for planning future 
services, as well as for managing support of this program. 

Enclosed is a questionnaire posing a variety of questions about your experience with NC LIVE, as well as 
with computers and electronic resources in general. Please look over the questionnaire. You signify your 
willingness to participate by completing and returning the survey in the enclosed stamped and self-addressed 
envelope no later than May 15, 1999. The results of this study will be summarized and published in 
aggregate form only. Your responses will be held in strictest confidence and will not be identified with you 
as an individual. 

Please take about ten minutes to answer this questionnaire and return it. Your willingness to share your 
experiences will be of value not only to my research, but may provide a better understanding of the public's 
use of NC LIVE as we come to the close of its first academic year. If you don’t feel you spend enough time 
at the reference desk to answer this survey, please pass it along to someone else on your staff. 

If you have questions or concerns about your rights as a research subject, you may contact my advisor. Dr. 
Barbara Wildemuth, at 962-0872 or email wildemuth@ils.unc.edu; or the UNC-CH Academic Affairs 
Institutional Review Board at the following addresses or telephone number. 

Academic Affairs Institutional Review Board, David A. Eckerman, Chair 

CB#4I00, 201 Bynum Hall 

The University of North Carolina at Chapel Hill 

Chapel Hill, North Carolina 27599-4100 

(919) 962-7761 

email: aa-irb-chair@unc.edu 

Thank you for your help. If you would like to receive a summary of the findings of this study, simply return 
the enclosed address form. 

Sincerely, 



Michele Matz 
M.S.L.S. Candidate 

School of Information and Library Science 
(919) 942-7828 



O 
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THE UNIVERSITY OF NORTH CAROLINA 
AT 

CHAPEL HILL 



School of Information and Library Science The University of North Carolina at Chapel Hill 

Phone# (919) 962-8366 CB# 3360, 100 Manning Hall 

Fax# (9 19) 962-8071 Chapel Hill, N.C. 27599-3360 

April 19, 1999 

Dear Academic Librarian: 

I am conducting a survey for my master’s thesis of the impact NC LIVE has on patrons’ use of electronic 
services in academic libraries. The objective of this research project is to understand the impact on academic 
library staffing of the public's increased access to electronic resources, as well as to study aspects of 
electronic research. Understanding the impact of programs like NC LIVE is important for planning future 
services, as well as for managing support of this program. 

I am administering this survey on a web page located at http://ils,unc,edu/iiiatz/NCLIVE/, where it will be 
posted until May The questionnaire poses a variety of questions about your experience with NC LIVE, 
as well as with computers and electronic resources in general. Please look it over. You signify your 
willingness to participate by completing and returning the survey electronically no later than May 7, 7999. 

The results of this project will be summarized and published in aggregate form only. Your responses will be 
held in strictest confidence and will not be identified with you as an individual. 

Please take about ten minutes to answer this questionnaire and return it electronically. Your willingness to 
share your experiences will be of value not only to my research, but may provide a better understanding of 
the public's use of NC LIVE as we come to the close of its first academic year. If you don’t feel you spend 
enough time at the reference desk to answer this survey, please pass this message along to someone else on 
your staff. 

If you have questions or concerns about your rights as a research subject, you may contact my advisor. Dr. 
Barbara Wildemuth, at 962-0872 or email wildemuth@ils.unc.edu; or the UNC-CH Academic Affairs 
Institutional Review Board at the following addresses or telephone number. 

Academic Affairs Institutional Review Board, David A. Eckerman, Chair 

CB# 4100, 201 Bynum Hall 

The University of North Carolina at Chapel Hill 

Chapel Hill, North Carolina 27599-4100 

(919) 962-7761 

email: aa-irb-chair@unc.edu 

Thank you for your help. If you would like to receive a summary of the findings of this study, simply return 
the enclosed address form. 

Sincerely, 



Michele Matz 
M.S.L.S. Candidate 

School of Information and Library Science 
(919) 942-7828 

O 
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April 20, 1999 



Dear Academic Librarian: 

I am conducting a survey for my master’s thesis of the impact NC LIVE has on patrons’ use of electronic 
services in academic libraries. The objective of this research project is to understand the impact on academic 
library staffing of the public's increased access to electronic resources, as well as to study aspects of 
electronic research. Understanding the impact of programs like NC LIVE is important for planning future 
services, as well as for managing support of this program. 

I am administering this survey on a web page located at http://ils.unc.edu/matz/survey/, where it will be 
posted until May The questionnaire poses a variety of questions about your experience with NC LIVE, 
as well as with computers and electronic resources in general. Please look it over. You signify your 
willingness to participate by completing and returning the survey electronically no later than May 7, 7999. 
The results of this project will be summarized and published in aggregate form only. Your responses will be 
held in strictest confidence and will not be identified with you as an individual. 

Please take about ten minutes to answer this questionnaire and return it electronically. Your willingness to 
share your experiences will be of value not only to my research, but may provide a better understanding of 
the public's use of NC LIVE as we come to the close of its first academic year. If you don’t feel you spend 
enough time at the reference desk to answer this survey, please pass this message along to someone else on 
your staff. 

If you have questions or concerns about your rights as a research subject, you may contact my advisor. Dr. 
Barbara Wildemuth, at 962-0872 or email wildemuth@ils.unc.edu; or the UNC-CH Academic Affairs 
Institutional Review Board at the following addresses or telephone number. 

Academic Affairs Institutional Review Board 

David A. Eckerman, Chair 

CB# 4100, 201 Bynum Hall 

The University of North Carolina at Chapel Hill 

Chapel Hill, North Carolina 27599-4100 

(919)962-7761 

email: aa-irb-chair@unc.edu 

Thank you for your help. If you would like to receive a summary of the findings of this study, simply return 
this email to me with your name and mailing address. 

Sincerely, 

Michele Matz 
M.S.L.S. Candidate 

School of Information and Library Science 
(919) 942-7828 
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Appendix D 

Selected Statistical Test Results 

One-way analysis of variance and chi-square tests were used to compare individual 
variables over the three survey groups. A Bonferroni post-hoc analysis was used to 
analyze the significant ANOVA. Spearman’s rho correlations were used to test 
relationships among ordinal data. Pearson correlations were used to test relationships 
among interval data. 

Tests for Hypothesis 1 

*** Hypothesis 1: Respondents to the Web survey will not exhibit significantly 
different demographics from respondents to the paper survey. 

Ns are presented for each test. 



Section III 

1. Age: 24 or below □ 25-34 □ 35-44 □ 45-54 □ 55-64 □ 65 or above □ 





Survey Grou 


ps 


Total 




paper 


email/web 


mail/web 






24 or below 


0 


0 


0 


0 




25-34 years 


8 


10 


2 


20 




35-44 years 


7 


6 


8 


21 




45-54 years 


29 


15 


15 


59 




55-64 years 


7 


9 


2 


18 




65 or above 


0 


0 


0 


0 


Total 




51 


40 


27 


N=118 


'=10.245, with 6 df, p=.l] 


15 



Section III 

2. Sex: Male □ Female □ 





Survey Grou 




Total 




paper 


email/web 


mail/web 






male 


13 


10 


10 


33 




female 


37 


30 


17 


84 


Total 




50 


40 


27 


N=117 



X^=1.36, with 2 df, p=.506 
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Section III 

4. Please check your highest level of library education completed: 

Paraprofessional library training □ Associate’s degree □ Bachelor’s degree □ 

M.L.S.Q Ph.D. □ 





Survey Grou 


ps 


Total 


Library education level 


paper 


email/web 


mail/web 






paraprof 


2 


4 


0 


6 




Assoc 


2 


0 


0 


2 




BA 


1 


0 


0 


1 




MLS 


46 


36 


27 


109 




PhD 


0 


0 


0 


0 


Total 




51 


40 


27 


N=118 


X^=7.57 with 6 c 


If, p=.271 



Section III 

8. Overall, how long have you worked full-time in libraries (this one plus any other libraries where 
you may have been employed full-time)? Years Months 





r 


P 


Time employed in libraries 


-.054 


.563 



(N=117) 



Section III 

10. How many hours each week do you work at the reference desk? 

5 or less □ 6-10 □ 11-15 □ 16-20 □ 21-30 □ 30 or more □ 





Survey Grou 


ps 


Total 




paper 


email/web 


mail/web 






<5 hours 


4 


8 


1 


13 




6-10 hours 


15 


6 


10 


31 




11-15 hours 


15 


14 


8 


37 




16-20 hours 


9 


2 


2 


13 




21-30 hours 


4 


3 


3 


10 




>30 hours 


4 


6 


3 


13 


Total 




51 


39 


27 


N=117 


;'=13.33, with 10 df, p=.2( 


06 
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Section III 

1 1 . Would you categorize your work as primarily 

public service (reference) □ primarily technical □ or primarily managerial □? 





Survey Grou 


ps 


Total 




paper 


email/web 


mail/web 






public service 


27 


28 


19 


74 




Technical 


13 


6 


1 


20 




managerial 


10 


6 


7 


23 


Total 




50 


40 


27 


N=117 



X^=7.420, with 4 df, p=.115 



Section III 

13. If you are currently working in an academic library, what is its Carnegie classification? 



□ Research University I 

□ Doctoral University II 

□ Baccalaureate (Liberal Arts) 
College I 

□ Professional School/Specialized 
Institution 



□ Research University II 

□ Master's (Comprehensive) 
University/College I 

□ Baccalaureate (Liberal Arts) 
College II 



□ Doctoral University I 

□ Master's (Comprehensive) 
University/College II 

□ Associate of Arts College 





Survey Groups 


Total 


Carnegie classification 


paper 


email/web 


mail/web 






Rsrchl 


2 


7 


2 


11 




Rsrch2 


0 


1 


0 


1 




Docl 


4 


2 


2 


8 




Doc 2 


5 


2 


2 


9 




MAI 


2 


6 


4 


12 




MA2 


4 


5 


0 


9 




BAl 


13 


3 


6 


22 




BA2 


7 


3 


1 


11 




AA 


13 


9 


8 


30 




PS/Sl 


0 


1 


0 


1 


Total 




50 


39 


25 


N=114 



x'=22.70 with 18 df, p=.203 




57 



D-4 



Section III 

15. If you are currently working in a academic library, what is the size of the library’s collection? 
volumes 



Mean Collection Size^ By Survey Medium 





Paper 


Web/Email 


Web/Mail 


F 


df 


P 


Collection size 


372,856.4 


892,068.3 


572,840.2 


2.668 


2 


.074 



Section III Yes □ No □ 

16. Do you have your own computer at home? 





Survey Grou 


ps 


Total 




paper 


email/web 


mail/web 






No 


15 


14 


9 


38 




Yes 


36 


26 


18 


80 


Total 




51 


40 


27 


118 



x'=3.41, with2df, p=.843 



Section III 

1 8. If you have a computer at home, do you have email 

access of any sort at home? Yes □ No □ 





Survey Grou 


ps 


Total 




paper 


email/web 


mail/web 






No 


7 


6 


2 


15 




Yes 


29 


20 


16 


65 


Total 




36 


26 


18 


N=80 



X^=1.02, with 2 df, p=.600 
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Section III 

20. Do you have Internet access at home? Yes □ No □ 





Survey Grou 


3S 


Total 




paper 


email/web 


mail/web 






No 


7 


6 


1 


14 




Yes 


29 


20 


17 


66 


Total 




36 


26 


18 


N=80 



x'=2.43, with 2 df, p=.296 



Tests for Hypotheses 2 and 3 

Hypothesis 2; Respondents to the Web survey will not give responses that are 
significantly different from responses to the paper survey. 

Hypothesis 3: Respondents will not provide significantly different 
answers to the Web survey whether they were notified of it by email or 
postal mail. 

To test these hypotheses, the content of the responses to several items in Sections I and II 
were compared. Ns for the ANOVA tests for these hypotheses varied between 109-118 
overall (47-51 for the paper survey, 32-40 for the Web survey/notified by email, and 18- 
27 for the Web survey/notified by mail). 



Section I 

2. a. If you answered yes to question I, how long has your library had NC LIVE? months 



Mean Number of Months, By Survey Medium 





Paper 


Web/Email 


Web/Mail 


F 


df 


P 


Number of Months 


11.67 


11.55 


11.41 


.118 


2 


.889 
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Section I 

3. In your view, what are the greatest benefits of having NC LIVE? Please check all that apply. 



□ Helps staff learn more electronic resources 

□ Encouragement of patrons to explore 
electronic resources 

□ Provides a marketing tool to promote the 
library to the community 



□ Greater variety of resources for patrons 

□ Brings more people into the library 

□ Other: 



Mean Proportion Checking Each Response Option, By Survey Medium 





Paper 


Web/Email 


Web/Mail 


F 


df 


P 


Staff learning 


.45 


.49 


.44 


.077 


2 


.926 


Resource variety 


.92 


.95 


1.00 


1.109 


2 


.334 


Patron exploration 


.63 


.44 


.63 


1.971 


2 


.144 


Increased patronage 


.29 


.31 


.19 


.690 


2 


.504 


Marketing tool 


.37 


.31 


.26 


.547 


2 


.580 



Section I 

4. In your view, what are the greatest detriments of having NC LIVE? Please check all that apply. 

□ Additional bibliographic instruction □ Increases patron reliance on electronic 

demands on staff resources even when print resources are 

□ Heavy demand for computers more appropriate 

□ Strains staff coverage at the reference desk □ Other: 



Mean Proportion Checking Each Response Option, By Survey Medium 





Paper 


Web/Email 


Web/Mail 


F 


df 


P 


Increased BI 


.51 


.46 


.56 


.282 


2 


.755 


Reliance on 
e- resources 


.69 


.56 


.52 


1.260 


2 


.288 


High computer 
demand 


.36 


.54 


.52 


1.681 


2 


.191 


Strained staff 
coverage 


.25 


.21 


.33 


.679 


2 


.509 
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Section I 

5. How many public computer stations in the library where you work give patrons access to NC 



LIVE? 



Mean Number of Computers, By Survey Medium 





Paper 


Web/Email 


Web/Mail 


F 


df 


P 


Number of 
Computers 


27.40 


45.97 


30.52 


2.454 


2 


.091 



Section I 

6. Can patrons access multiple software applications on these computer stations, including software 
like word-processing programs? Yes □ No □ 





Survey Grou 


ps 


Total 


Access multiple 
software 


paper 


email/web 


mail/web 






No 


10 


6 


10 


26 




Yes 


41 


33 


17 


91 


Total 




51 


39 


27 


N=117 



xM.69, 2 df, p=.096 



Section I 

8. Assess the level of demand for the library’s public computer stations before NC LIVE was 
available (1 is least; 5 is greatest). 



1 2 3 

less than 5 
users per hour 
at peak periods 



more than 20 
users per hour 
at peak periods 



Mean Demand, By Survey Medium 





Paper 


Web/Email 


Web/Mail 


F 


df 


P 


Demand Pre-NC LIVE 


3.28 


4.00 


3.56 


3.095 


2 


.049 



Bonferroni 







Mean 

Difference 

(I-J) 


Std. Error 


Sig. 


95% 

Confidence 

Interval 




(I) Survey 
Group 


(J) Survey 
Group 








Lower 

Bound 


Upper 

Bound 


paper 


email/web 


-.72 


.29 


.043 


-1.43 


-1.55E-02 


paper 


mail/web 


-.28 


.31 


1.000 


-1.04 


.49 


email/web 


mail/web 


.44 


.33 


.558 


-.37 


1.26 
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Section I 

9. Assess the level of demand for these stations now with NC LIVE (1 is least; 5 is greatest). 



1 2 
less than 5 
users per hour 
at peak periods 



more than 20 
users per hour 
at peak periods 



Mean Demand, By Survey Medium 





Paper 


Web/Email 


Web/Mail 


F 


df 


P 


Demand Now 


4.00 


4.40 


4.35 


1.998 


2 


.141 



Section I 

10. Assess your own frequency of use of electronic indexes either on CD-ROM or via the Internet 
(e.g., ERIC, Psychlit, MEDLINE) per hour at the reference desk prior to gaining NC LIVE (1 is 
least; 5 is greatest). Do not include uses of the library’s online catalog. 



1 

less than 2 
uses per hour 


2 

Mean Free 


3 

uency of Use, By Survey Medium 


4 


5 

more than 10 
uses per hour 




Paper 


Web/Email 


Web/Mail 


F 


df 


P 


Frequency before 
NC LIVE 


2.66 


3.00 


2.56 


.990 


2 


.375 



Section I 

11. Assess your own frequency of use of electronic indexes through NC LIVE per hour at the 
reference desk (1 is least; 5 is greatest). 

1 2 3 4 5 

less than 2 more than 1 0 

uses per hour uses per hour 



Mean Frequency of Use, By Survey Medium 





Paper 


Web/Email 


Web/Mail 


F 


df 


P 


Current frequency 


3.12 


3.39 


3.07 


.658 


2 


.520 



Section I 

13. How many databases in NC LIVE do you personally use regularly? 



Mean Number of Databases, By Survey Medium 





Paper 


Web/Email 


Web/Mail 


F 


df 


P 


Number of Databases 
Used 


6.40 


5.31 


4.74 


1.065 


2 


.348 
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Section I 

15. Has NC LIVE increased the amount of bibliographic instruction you provide to patrons? 





Survey Grou 


ps 


Total 


Access multiple 
software 


paper 


email/web 


mail/web 






No 


7 


9 


4 


20 




Yes 


44 


26 


23 


93 


Total 




51 


35 


27 


N=113 



x'=5.40, 4 df, p=.248 



Section I 

17. How much in-class training have you had in using NC LIVE (in hours)? 



Mean Number of Hours, By Survey Medium 





Paper 


Web/Email 


Web/Mail 


F 


df 


P 


Training hours 


6.44 


6.72 


7.32 


.092 


2 


.912 



Section I 

18. Do you feel these workshops have provided sufficient training? Yes □ No □ 





Survey Grou 


3S 


Total 


Access multiple 
software 


paper 


email/web 


mail/web 






No 


8 


4 


6 


18 




Yes 


31 


20 


16 


68 


Total 




39 


25 


22 


86 



x'=3.26,4df,p=.515 
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Section II 

8. Which unit below best describes the frequency with which you have referred to electronic 

literature databases, either on CD-ROM or via the Internet, during reference desk work in the last 
year? 

Every hour □ Every day □ Every week □ Every month □ 





Survey Grou 


)S 


Total 


Frequency last year 


paper 


email/web 


mail/web 






everyhr 


0 


1 


0 


1 




everyday 


5 


2 


0 


7 




everywk 


23 


14 


9 


46 




every mo 


23 


20 


16 


59 


Total 




51 


37 


25 


N=113 



x'=6.35, 6 df, p=.385 



Section II 

9. Which unit below best describes the frequency with which you currently use such databases 
during reference desk work? 

Every hour □ Every day □ Every week □ Every month □ 





Survey Grou 


ps 


Total 


Frequency currently 


paper 


email/web 


mail/web 






everyhr 


1 






1 




everyday 


5 


2 


1 


8 




everywk 


16 


11 


7 


34 




everymo 


29 


24 


17 


70 


Total 




51 


37 


25 


N=113 



x'=6.09, 8 df, p=.637 
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Section II 

Please estimate the number of times you use each of the following electronic resources at work but outside 
of reference desk service in a typical week: 





Times Per Week 


Never 


Less than 10 


11-25 


26-50 


More than 
50 


10. World Wide Web access 

(e,g,, research or surfing) 















Survey Grou 


ps 


Total 


Frequency of WWW use 


paper 


email/web 


mail/web 






never 


0 


0 


0 


0 




<10 


13 


5 


3 


21 




11-25 


19 


13 


14 


46 




26-50 


12 


11 


5 


28 




>50 


6 


11 


5 


22 


Total 




50 


40 


27 


N=117 



x'=8.13, with6df, p=229 



Section II 

Please estimate the number of times you use each of the following electronic resources at work but outside 
of reference desk service in a typical week: 







Times Per Week 






Never 


Less than 10 


11-25 


26-50 


More than 
50 


11. 


Electronic mail 















Survey Grou 


ps 


Total 


Frequency of email use 


paper 


email/web 


mail/web 






never 


0 


0 


1 


1 




<10 


9 


5 


1 


15 




11-25 


17 


15 


9 


41 




26-50 


13 


8 


7 


28 




>50 


12 


12 


9 


33 


Total 




51 


40 


27 


N=118 



X^=7.26 with 8 df, p=.509 
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Section II 

Please estimate the number of times you use each of the following electronic resources at work but outside 
of reference desk service in a typical week: 







Times Per Week 






Never 


Less than 10 


11-25 


26-50 


More than 
50 


12. 


Listservs/Newsgroups 















Survey Grou 


3S 


Total 


Frequency of 
listservs/newsgroups use 


paper 


email/web 


mail/ web 






never 


7 


6 


1 


14 




<10 


20 


18 


14 


52 




11-25 


14 


7 


5 


26 




26-50 


5 


7 


3 


15 




>50 


3 


2 


3 


8 


Total 




49 


40 


26 


N=115 



X^=5.94 with 8 df, p=.654 



Section II 

Please estimate the number of times you use each of the following electronic resources at work but outside 
of reference desk service in a typical week: 





Times Per Week 


Never 


Less than 10 


11-25 


26-50 


More than 
50 


13. Searching or maintaining 
your library’s online catalog 















Survey Grou 


ps 


Total 


Frequency of online 
catalog use 


paper 


email/web 


mail/web 






never 


2 


0 


0 


2 




<10 


10 


6 


9 


25 




11-25 


13 


8 


3 


24 




26-50 


12 


11 


9 


32 




>50 


14 


13 


6 


33 


Total 




51 


38 


27 


N=116 



x'=8.21 with 8df,p=.413 
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Section II 

Please estimate the number of times you use each of the following electronic resources at work but outside 
of reference desk service in a typical week: 





Times Per Week 


Never 


Less than 10 


1 1 -25 


26-50 


More than 
50 


14. Computing software 
word-processing or 
spreadsheet programs) 















Survey Grou 


ps 


Total 


Frequency of computing 
software use 


paper 


email/web 


mail/web 






never 


2 


1 




3 




<10 


17 


12 


8 


37 




11-25 


12 


17 


12 


41 




26-50 


12 


6 


2 


20 




>50 


8 


4 


5 


17 


Total 




51 


40 


27 


N=118 



x'=8.21 with 8 df,p=.413 



Tests for Hypothesis 5 

<♦ Hypothesis 5: At least 50 percent of the total number of responses to the Web 
survey will be returned in one week. 

Date Responses Were Received 



Mean Date Received, By Survey Medium 





Paper 


Web/Email 


Web/Mail 


F 


df 


P 


Date 


13.41 


4.20 


6.74 


37.769 


2 


.000 



Bonferroni 







Mean 

Difference 

(I-J) 


Std. Error 


Sig. 


95% 

Confidence 

Interval 




(I) Survey 
Group 


(J) Survey 
Group 








Lower 

Bound 


Upper 

Bound 


paper 


email/web 


9.21 


1.10 


.000 


6.54 


11.88 


paper 


mail/web 


6.67 


1.24 


.000 


3.66 


9.68 


email/web 


mail/web 


-2.54 


1.30 


.157 


-5.69 


.61 



O 

ERIC 
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Other Comparisons 

A few significant relationships appeared in comparisons of variables other than 
those between the different survey groups. Librarians in lower Carnegie classification 
institutions indicated having a greater number of training hours and a greater desire for 
additional training than those in research and doctoral institutions. There was a strong 
negative correlation between the Carnegie classification of the respondents’ libraries and 
the number of training hours respondents received (Spearman’s rho =-0.354, p= 0.000). 
The higher the Carnegie classification, the lower the number of training hours the 
respondent was likely to indicate having received. The correlation between the number 
of computers and the number of training hours also showed a strong negative relationship 
(Pearson’s r=-0.258, p=0.008). (The relationship between the Carnegie classification 
and number of computers is, of course, a strong positive one [Spearman’s rho=.715, 

p=0.000].) 

This is logical because libraries in research institutions with greater resources 
often already had access to and experience with electronic databases in general prior to 
the inception of the NC LIVE program, including some specific databases to which NC 
LIVE provides access. This also explains why the number of computers in the 
respondents’ libraries was also positively correlated with high demand for computers 
both before and after the inception of the NC LIVE program (Pearson’s r=0.516, 
p=0.000), and with the likelihood the respondent would report a high level of use of 
electronic resources prior to the inception of NC LIVE (Pearson’s r=0.289, p=0.003). 

The number of computers also exhibited a significant positive correlation with 
respondents’ more frequent use of listservs and news groups (Pearson’s r=0.193. 



O 

ERIC 
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p=0.043). Although the relationships between the number of computers and more 
frequent use of the Web or email or other electronic resources were not significant, these 
relationships showed the same positive trend. 

Interestingly, respondents who reported an increase in bibliographic instruction in 
general, and specifically those who reported an increase in time spent with individual 
patrons, tended to have a smaller number of computers (t=^.164 with df=107, p=0.000, 
and t=-3.391 with df=104, p=0.000 respectively). This could be explained, however, by 
the possibility that libraries in research institutions may have done more of this type of 
instruction than smaller libraries prior to the introduction of NC LIVE. Consequently, 
they would not have experienced an increase with the inception of NC LIVE because 
instruction would already have been occurring frequently. 
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Survey Content Summary 
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Section I. NC LIVE 

1. If you are currently employed in an academic library, does it have NC LIVE 
access? Yes - 118 No - 0 

All 118 libraries reported having access to NC LIVE. No respondents answered that their 
library did not have NC LIVE access. There were 51 responses from the paper survey, 

40 from the Web survey group notified by email, and 27 from the Web survey group 
notified by mail. 



2. a. If you answered yes to question 1, how long has your library had NC 
LIVE? 

months 

b. If you answered no to question 1, do you know when you will get access to 

it? (month, year) —Please proceed to Section II, at the bottom of 

the next page. 

Most libraries (58.5%) had been able to access NC LIVE for one year; only five 
respondents reported having access for less than seven months. 



Number of 
Months with 
Access 


Total Number 
of Responses 


Responses from the 
Paper Survey 


Responses from 
the Web 
Survey/Email 
Notice 


Responses from 
the Web 
Survey/Mail 
Notice 


4 


1 


1 


0 


0 


6 


3 


2 


0 


1 


7 


1 


0 


0 


1 


8 


3 


2 


0 


1 


9 


5 


1 


3 


1 


10 


12 


3 


5 


4 


11 


11 


7 


2 


2 


12 


69 


27 


28 


14 


13 


6 


5 


1 


0 


14 


2 


0 


1 


1 


16 


2 


1 


0 


1 


18 


2 


1 


0 


1 


24 


1 


1 


0 


0 



(N=118) 
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3. In your view, what are the greatest benefits of having NC LIVE? Please 
check all that apply. 

Respondents most often cited the advantage of the greater variety of resources libraries 
could offer patrons via NC LIVE. 



Benefits of NC LIVE: 


#of 

Responses 


Paper 

Survey 


Web Survey/ 
Email Notice 


Web Survey/ 
Mail Notice 


Helps staff learn more electronic 
resources 


54 


23 


19 


12 


Greater variety of resources for 
patrons 


111 


47 


37 


27 


Encouragement of patrons to 
explore electronic resources 


66 


32 


17 


17 


Brings more people into the 
library 


32 


15 


12 


5 


Provides a marketing tool to 
promote the library to the 
community 


38 


19 


12 


7 


Other 


36 


10 


16 


10 



(N=117) 



4. In your view, what are the greatest detriments of having NC LIVE? 
Please check all that apply. 

The disadvantage most frequently cited was the increased reliance of patrons on 
electronic resources even when print resources were more appropriate. Libraries with 
lower Carnegie classifications were less likely to have this concern. 



Disadvantages of NC LIVE: 


#of 

Responses 


Paper 

Survey 


Web Survey/ 
Emai] Notice 


Web Survey/ 
Mail Notice 


Additional bibliographic 
instruction demands on staff 


59 


26 


18 


15 


Increases patron reliance on 
electronic resources even when 
print resources are more 
appropriate 


71 


35 


22 


14 


Heavy demand for computers 


53 


18 


21 


14 


Strains staff coverage at the 
reference desk 


30 


13 


8 


9 


Other 


32 


9 


9 


14 



(N=117) 
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5. How many public computer stations in the library where you work give 
patrons access to NC LIVE? 

The number of computers varied greatly. 15 respondents reported their libraries had 
more than 100 computers; 45 reported having less than 20. The most frequently reported 
numbers were 100 and 20 (8 respondents each), followed by 8 (7 respondents). Nine, 
fourteen and fifteen computers were reported by six respondents each. (N=l 13) 



Number of 
Computers 


Total Number of 
Responses 


Paper Survey 


Web Survey/Email 
Notice 


Web Survey/Mail 
Notice 


2 


3 


1 


1 


0 


3 


1 


0 


0 


1 


4 


2 


2 


0 


0 


5 


2 


1 


0 


1 


6 


4 


2 


1 


1 


7 


5 


4 


1 


0 


8 


7 


4 


1 


2 


9 


6 


3 


1 


2 


10 


5 


2 


1 


2 


11 


2 


2 


0 


0 


12 


2 


1 


0 


1 


14 


6 


4 


2 


0 


15 


6 


5 


1 


0 


16 


2 


1 


1 


0 


17 


2 


0 


1 


1 


18 


3 


1 


2 


0 


19 


3 


1 


0 


2 


20 


8 


3 


4 


1 


21 


1 


0 


1 


0 


22 


1 


0 


0 


1 


24 


1 


0 


0 


1 


25 


1 


0 


0 


1 


26 


1 


0 


1 


0 


27 


3 


1 


1 


1 


30 


2 


1 


1 


0 


32 


1 


0 


0 


1 


34 


1 


0 


0 


1 


35 


1 


0 


1 


0 


36 


3 


1 


2 


0 


40 


3 


1 


1 


1 


45 


4 


1 


2 


1 


49 


1 


0 


1 


0 


50 


1 


1 


0 


0 
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Number of 
Computers 


Total Number of 
Responses 


Paper Survey 


Web Survey/Email 
Notice 


Web Survey/Mail 
Notice 


58 


1 


0 


1 


0 


60 


1 


0 


1 


0 


70 


1 


0 


0 


1 


75 


2 


0 


0 


2 


100 


7 


1 


5 


2 


130 


2 


1 


1 


0 


135 


1 


1 


0 


0 


140 


1 


1 


0 


0 


170 


1 


1 


0 


0 


175 


1 


0 


1 


0 


200 


1 


0 


1 


0 



6. Can patrons access multiple software applications on these computer 
stations, including software like word-processing programs? 

77.8% of respondents indicated their libraries offered patrons access to multiple software 
programs (N=l 17). Several who responded to the paper survey noted that software was 
available in separate computer labs in the library, in which case they were included in the 
“Yes” group. Some indicated such labs were available elsewhere on campus, in which 
case they were included in the “No” group. If web survey respondents had been able to 
comment, some of them might have indicated similar circumstances. 





Total Number 
of Responses 


Paper Survey 


Web Survey/ 
Email Notice 


Web Survey/ 
Mail Notice 


No 


10 


6 


10 


26 


Yes 


41 


33 


17 


91 
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7. If yes, please check software applications available. Please check all that 
apply. 

The most commonly cited “Other” software was stand-alone CD-ROMs. I expect the 
number of libraries offering these might have been higher if this had been a separate 
option. 



Offer: 


Total 

Number of 
“Yes” 
Responses 


Paper 

Survey 


Web 

Survey/Email 

Notice 


Web 

Survey/Mail 

Notice 


Word-processing? 


43 


20 


14 


9 


Spreadsheet 

programs? 


31 


16 


9 


6 


Library OP AC? 


80 


33 


32 


15 


Scanning software? 


10 


5 


4 


1 


Internet browsers? 


93 


40 


35 


18 


Other? 


27 


II 


9 


7 



(N=114) 



8. Assess the level of demand for the library’s public computer stations before 
NC LIVE was available (1 is least; 5 is greatest). 



1 2 3 

less than 5 
users per hour 
at peak periods 



more than 20 
users per hour 
at peak periods 



Most libraries ranked the demand for computers prior to receiving NC LIVE quite high. 



Demand Level 
Prior to 
Receiving NC 
LIVE Access 


Total Number 
of Responses 


Paper Survey 


Web Survey/ 
Email Notice 


Web Survey/ 
Mail Notice 


I 


8 


4 


I 


3 


2 


18 


10 


4 


4 


3 


26 


14 


7 


5 


4 


17 


7 


5 


5 


5 


40 


12 


18 


10 



(N=109) 
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9. Assess the level of demand for these stations now with NC LIVE (1 is least; 5 is 
greatest). 



1 2 3 

less than 5 
users per hour 
at peak periods 



more than 20 
users per hour 
at peak periods 



Few libraries indicated a change in demand from lesser to greater after receiving NC 
LIVE access. 



Demand Level After 
Receiving NC LIVE Access 


Total Number 
of Responses 


Paper Survey 


Web 

Survey/Email 

Notice 


Web Survey/Mail 
Notice 


1 


2 


0 


1 


1 


2 


4 


3 


1 


0 


3 


19 


12 


4 


3 


4 


27 


14 


6 


7 


5 


56 


18 


23 


15 



(N=108) 



10. Assess your own frequency of use of electronic indexes either on CD-ROM or 
via the Internet (e.g., ERIC, Psychlit, MEDLINE) per hour at the reference 
desk prior to gaining NC LIVE (1 is least; 5 is greatest). Do not include uses of 
the library’s online catalog. 

1 2 3 4 5 

less than 2 more than 10 

uses per hour uses per hour 



Frequency of Use of 
Electronic Databases Prior to 
NC LIVE Access 


Total Number 
of Responses 


Paper Survey 


Web 

Survey/Email 

Notice 


Web 

Survey/Mail 

Notice 


1 


27 


12 


6 


9 


2 


19 


7 


6 


6 


3 


30 


21 


6 


3 


4 


22 


6 


10 


6 


5 


11 


4 


4 


3 



(N=109) 
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11. Assess your own frequency of use of electronic indexes through NC LIVE per 

hour at the reference desk (1 is least; 5 is greatest). 



1 2 3 4 5 

less than 2 more than 10 

uses per hour uses per hour 



Frequency of Use of Electronic 
Databases through NC LIVE 


Total Number 
of Responses 


Responses from 
the Paper 
Survey 


Responses from 
the Web 
Survey/Email 
Notice 


Responses from 
the Web 
Survey /Mail 
Notice 


1 


11 


4 


4 


3 


2 


25 


10 


6 


9 


3 


27 


17 


6 


4 


4 


31 


14 


12 


5 


5 


19 


5 


8 


6 



(N=113) 



12. Which resources within NC LIVE do you refer patrons to most frequently? 

The two databases to which respondents most frequently referred patrons were the full- 
text resources, ProQuest and Ebscohost, by a wide margin. ProQuest was cited eighty- 
one times and Ebscohost sixty-five times. OCLC FirstSearch was the next closest choice, 
with thirty-one occurrences. Respondents mentioned Psycinfo eighteen times and ERIC 
eleven times. Others that had multiple votes included Medline, with six, and InfoTrac, 
with two. 



ERIC 
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13. How many databases in NC LIVE do you personally use regularly? 

There was a wide range in the number of databases used regularly, but the majority of 
respondents reported regularly using three. 



Number of NC LIVE 
Databases Used Regularly 


Total Number 
of Responses 


Responses 
from the Paper 
Survey 
(Number) 


Responses from 
the Web 
Survey/Email 
Notice (Number) 


Responses from 
the Web 
Survey/Mail 
Notice (Number) 


0 


1 


1 


0 


0 


1 


3 


2 


0 


1 


2 


12 


4 


5 


3 


3 


24 


6 


14 


4 


4 


16 


6 


3 


7 


5 


17 


10 


3 


4 


6 


14 


7 


3 


4 


7 


2 


0 


1 


1 


8 


2 


1 


1 


0 


9 


2 


2 


0 


0 


10 


10 


4 


3 


3 


12 


1 


1 


0 


0 


14 


2 


1 


1 


0 


15 


1 


0 


1 


0 


20 


1 


0 


1 


0 


25 


1 


1 


0 


0 


40 


1 


1 


0 


0 



(N=110) 



14. Which ones do you use most frequently? 

Respondents did not indicate they used noticeably different databases than the ones to 
which they refer patrons. Again, ProQuest and Ebscohost, along with WorldCat, were 
the most frequently used by respondents themselves. ProQuest was cited eighty-four 
times, WorldCat seventy-three times, and Ebscohost forty-three times. Psycinfo received 
sixteen votes, and ERIC nine. Medline again received six, and InfoTrac two. 
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15. Has NC LIVE increased the amount of bibliographic instruction you provide 
to patrons? 

NC LIVE increased bibliographic instruction for 75.41% of respondents. (N=l 13) 





Total Number 
of Responses 


Paper Survey 


Web Survey/ 
Email Notice 


Web Survey/ 
Mail Notice 


No 


7 


9 


4 


20 


Yes 


44 


26 


23 


93 



16. If yes, how? Please check all that apply. 

Time spent with individual patrons was cited most often. “Other” ways BI increased that 
were mentioned were in the amount of time spent with faculty and staff; the length of 
classes taught; and sessions formulated for particular classes. 



Ways BI Increased 


Total Number 
of “Yes” 
Responses 


Paper Survey 


Web Survey/Email 
Notice 


Web Survey/Mail 
Notice 


Time Spent with 
Individual 
Patrons 


88 


43 


23 


22 


# of Group 
Classes Taught 


45 


22 


9 


14 


Other 


10 


3 


4 


3 



(N=109) 



17. How much in-class training have you had in using NC LIVE (in hours)? 

Number of in-class training hours varied widely, from a minimum of zero hours to a 
maximum of 40 hours (some respondents to the paper survey indicated they participated 
in “train-the-trainer” sessions, accounting for the higher numbers of training hours). 

Many respondents (24.6%) had no training specific to NC LIVE. This group 
corresponded strongly with higher Carnegie classifications (Pearson’s r=-0.356, 
p=0.000). 21.4% of respondents had between 2 and 4 hours of training. 38.2% had more 
than 4 hours of training. The mean number of hours was 6.74, s.e.=8.24. 
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18. Do you feel these workshops have provided sufncient training? Yes □ No □ 

Most who had training felt the classes had not provided sufficient training. Most who 
had no training indicated they were satisfied they had enough training. These results are 
not too surprising when one considers that many of the larger libraries had access to 
many of the NC LIVE databases via independent licenses prior to the inception of the NC 
LIVE program. 





Total Number of 
Responses 


Paper Survey 


Web Survey/Email 
Notice 


Web Survey/Mail 
Notice 


No 


18 


8 


4 


6 


Yes 


67 


31 


20 


16 



(N=86) 



Section II. Electronic Library Services/Experience 

Please indicate whether the library where you work offers patrons the following 
kinds of access: 

Respondents’ libraries offer the following resources in the indicated numbers: 





Total Number 
of “Yes” 
Responses 


Paper Survey 


Web Survey/ 
Email 


Web Survey/ 
Mail 


1. Graphics-based World Wide 
Web searching/browsing {e.g., 
using Netscape, Mosaic or 
Explorer software) 


117 


51 


40 


26 


2. Text-based World Wide Web 
browsing {e.g., using Lynx 
software)* 


31 


11 


10 


10 


3. Email accounts 


66 


27 


26 


13 


4. Newsgroup services {e.g., access 
to newsgroup readers) * 


36 


12 


16 


8 


5. Library’s catalog online 


115 


49 


39 


27 


6. Computing software {e.g., word- 
processing or spreadsheet 
programs) 


65 


28 


20 


17 


7, Other 


31 


11 


8 


12 



(N=118) 



*Since most graphics-based web browsers are also capable of text-browsing and 
newsgroup services can be accessed via web browsers and email, it seems likely that 
many respondents did not fully understand these options. 
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8. Which unit below best describes the frequency with which you have referred to 
electronic literature databases, either on CD-ROM or via the Internet, during 
reference desk work in the last year? 

Every hour □ Every day □ Every week □ Every month □ 

Frequency of use of electronic databases overall in the last year was relatively low. 





Total Number 
of Responses 


Paper Survey 


Web Survey/ 
Email 


Web Survey/ 
Mail 


Every hour 


1 


0 


1 


0 


Every day 


7 


5 


2 


0 


Every week 


46 


23 


14 


9 


Every month 


59 


23 


20 


16 



(N=113) 



9. Which unit below best describes the frequency with which you currently use 
such databases during reference desk work? 

Every hour □ Every day □ Every week □ Every month □ 

Use at the current time does not show noticeable change from last year. 





Total Number 
of Responses 


Number from 
Paper Survey 


Number from 
Web Survey/ 
Email 


Number from 
Web Survey/ 
Mail 


Every hour 


1 


1 


0 


0 


Every day 


8 


5 


2 


1 


Every week 


34 


16 


11 


7 


Every month 


69 


29 


24 


16 



(N=112) 



Please estimate the number of times you use each of the following electronic 
resources at work but outside of reference desk service in a typical week: 



10. World Wide Web 


Total Number of 
Responses 


Paper Survey 


Web Survey/ 
Email 


Web Survey/ 
Mail 


Never 


0 


0 


0 


0 


Less than 10 


46 


19 


13 


14 


11-25 


28 


12 


11 


5 


26-50 


22 


6 


11 


5 


More than 50 


21 


13 


5 


3 



(N=117) 



er|c 
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11. Email 


Total Number of 
Responses 


Paper Survey 


Web Survey/ 
Email 


Web Survey/ 
Mail 


Never 


1 


0 


0 


i 


Less than 10 


15 


9 


5 


1 


11-25 


41 


17 


15 


9 


26-50 


28 


13 


8 


7 


More than 50 


33 


12 


12 


9 



(N=118) 



12. Listservs or Newsgroups 


Total Number 
of Responses 


Paper Survey 


Web Survey/ 
Email 


Web Survey/ 
Mail 


Never 


14 


7 


6 


1 


Less than 10 


52 


20 


18 


14 


11-25 


26 


14 


7 


5 


26-50 


15 


5 


7 


3 


More than 50 


8 


3 


2 


3 



(N=115) 



13. Online Catalog 


Total Number 
of Responses 


Paper Survey 


Web Survey/ 
Email 


Web Survey/ 
Mail 


Never 


2 


2 


0 


0 


Less than 10 


25 


10 


6 


9 


11-25 


24 


13 


8 


3 


26-50 


32 


12 


11 


9 


More than 50 


33 


14 


13 


6 



(N=116) 



14. Computing Software 


Total Number 
of Responses 


Paper Survey 


Web Survey/ 
Email 


Web Survey/ 
Mail 


Never 


3 


2 


1 


0 


Less than 10 


37 


17 


12 


8 


11-25 


41 


12 


17 


12 


26-50 


20 


12 


6 


2 


More than 50 


17 


8 


4 


5 



(N=118) 



15. Other Electronic 
Resources* 


Total Number 
of Responses 


Paper Survey 


Web Survey/ 
Email 


Web Survey/ 
Mail 


Never 


2 


1 


0 


1 


Less than 10 


2 


1 


1 


0 


11-25 


4 


3 


0 


1 


26-50 


2 


1 


0 


1 


More than 50 


3 


0 


3 


0 



(N=13) 




*Most frequently cited other resource was databases on CD ROM; others mentioned were technical 
reports, acquisitions database, OCLC/Interlibrary Loan, and desktop publishing applications. 



Section III. Demographics 

1. Age 

Participants’ age groups broke down as follows: 



E-13 



Age 


Total Number 
of Responses 


Paper Survey 


Web Survey/ 
Email 


Web Survey/ 
Mail 


25-34 years 


20 


8 


10 


2 


35-44 years 


21 


7 


6 


8 


45-54 years 


59 


29 


15 


15 


55-64 years 


18 


7 


9 


2 



(N=118) 



2. Sex 

Sex of participants broke down as follows: 



Sex 


Total 

Number of 
Responses 


Paper 

Survey 


Web Survey/ 
Email 


Web Survey/ 
Mail 


male 


33 


13 


10 


IQ 


female 


84 


37 


30 


17 



(N=117) 



3. Is English your first language? Yes □ No □ 

If no, what is your first language? 

Only two respondents indicated that their first language was not English. Both completed 
the paper survey, and neither specified their first language. (N=l 18) 



4. Please check your highest level of library education completed: 

Education level of participants broke down as follows: 



Education Level 


Total Number of 
Responses 


Paper Survey 


Web Survey/ 
Email 


Web Survey/ 
Mail 


Paraprofessional 


6 


2 


4 


0 


Associate’s 

degree 


2 


2 


0 


0 


Bachelor’s 

degree 


1 


1 


0 


0 


Master’s degree 


109 


46 


36 


27 



(N=118) 
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5. Please list any other non-library degrees you hold. 

Thirty-five respondents indicated they held graduate degrees in fields other than library 
science. Four respondents indicated they held degrees in other fields, but did not indicate 
at what level (whether bachelor’s or graduate). Twenty-three indicated they held 
bachelor’s (and in one case, associate’s) degrees in other fields. One person noted that 
they held a masters degree from a non-accredited institution. Disciplines most frequently 
mentioned were English and History. There were 37 responses from the paper survey, 14 
from the Web survey group notified by email, and 14 from the Web survey group notified 
by mail. (N=65) 



6. Are you currently employed more than 30 hours in a academic library? 
Yes □ No □ 



94.02% of respondents indicated they were employed full-time in an academic library. 





Total Number of 
Responses 


Paper Survey 


Web Survey/Email 
Notice 


Web Survey/Mail 
Notice 


No 


7 


4 


2 


1 


Yes 


no 


46 


38 


26 



(N=117) 



7. If so, how long have you worked in that library? Years Months 

Time employed in the current library varied from 2 months to 37 years, with an average 
of 10 and three-quarters years and a mode of 20 years. There were 48 responses from the 
paper survey, 40 from the Web survey group notified by email, and 27 from the Web 
survey group notified by mail. The standard deviation was 8.99. (N=115) 



8. Overall, how long have you worked full-time in libraries (this one plus any other 
libraries where you may have been employed full-time)? Years Months 

Time employed in libraries overall varied from 2 months to 41 years, with an average of 
16.1 years and a mode, again, of 20 years. There were 51 responses from the paper 
survey, 39 from the Web survey group notified by email, and 27 from the Web survey 
group notified by mail. The standard deviation was 9.70. (N=l 17) 



O 
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9. What is your job title? 

Job titles, not surprisingly, were all over the map. “Reference” and “Public Services” 
figured in most frequently with 35 occurrences, or 28.69%, between the two, but titles 
varied from A/V Librarian to Government Documents Librarian to simply Director. 
There were 50 responses from the paper survey, 38 from the Web survey group notified 
by email, and 27 from the Web survey group notified by mail. (N=l 15) 



10. How many hours each week do you work at the reference desk? 

Most people indicated working six to fifteen hours on the reference desk each week. 



Hours Worked Per 
Week 


Total Number of 
Responses 


Paper Survey 


Web Survey/ 
Email 


Web Survey/ Mail 


Less than 5 hours 


13 


4 


8 


1 


6-10 hours 


31 


15 


6 


10 


11-15 hours 


37 


15 


14 


8 


16-20 hours 


13 


9 


2 


2 


21-30 hours 


10 


4 


3 


3 


More than 30 hours 


13 


4 


6 


3 



(N=117) 



11. Would you categorize your work as primarily public service (reference), 
primarily technical, or primarily managerial? 

The majority of respondents indicated their primary work responsibility was public 
service. 



Type of Work 


Total Number 
of Responses 


Paper Survey 


Web Survey/ 
Email 


Web Survey/ 
Mail 


Public Service 


74 


27 


28 


19 


Technical 


20 


13 


6 


1 


Managerial 


23 


10 


6 


7 



(N=117) 
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12. If you answered ’’public service” above, do you mainly provide service to 
students, faculty or members of the larger community? 

The largest audience indicated by far was students. (N=83) 



Audience 


Total Number 
of Responses 


Number from 
Paper Survey 


Number from 
Web Survey/ 
Email 


Number from 
Web Survey/ 
Mail 


students 


80 


34 


29 


17 


faculty 


1 


0 


1 


0 


larger community 


2 


1 


0 


1 



13. If you are currently working in an academic library, what is its Carnegie 
classification? 



□ Research University I 

□ Doctoral University II 

□ Baccalaureate (Liberal Arts) 
College I 

□ Professional School/Specialized 
Institution 



□ Research University II 

□ Master’s (Comprehensive) 
University/College I 

□ Baccalaureate (Liberal 
Arts) College II 



□ Doctoral University I 

□ Master’s (Comprehensive) 
University/College II 

□ Associate of Arts College 



The Carnegie Classification of Higher Education groups American colleges and 
universities according to their missions. It aims to cluster institutions with similar 
programs and purposes. The largest number of respondents were in the Associate of Arts 
College classification, the category that includes most community colleges. (N=114) 



Classification 


Total Number of 
Responses 


Number from 
Paper Survey 


Number from 
Web Survey/ 
Email 


Number from 
Web Survey/ 
Mail 


Research University I 


11 


2 


7 


2 


Research University II 


1 


0 


1 


0 


Doctoral University I 


8 


4 


2 


2 


Doctoral University II 


9 


5 


2 


2 


Master’s (Comprehensive) 
University/College I 


12 


2 


6 


4 


Master’s (Comprehensive) 
University/College II 


9 


4 


5 


0 


Baccalaureate (Liberal 
Arts) College I 


22 


13 


3 


6 


Baccalaureate (Liberal 
Arts) College II 


11 


7 


3 


1 


Associate of Arts College 


30 


13 


9 


8 


Professional 

School/Specialized 

Institution* 


1 


0 


1 


0 



*Includes Religion and Theology; Medical; Health Professions; Engineering and Technology; Business and 
Management; Art, Music, and Design; Law; Teachers Colleges; Other Specialized Institutions; and Tribal 
Colleges and Universities. 
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14. If you are currently working in an academic library, is it the main campus 
library or a departmental library (health sciences, law, art, etc.)? 

Most respondents indicated working in the main library on their campus. 





Total 

Number of 
Responses 


Paper 

Survey 


Web Survey/ 
Email 


Web Survey/ 
Mail 


Departmental 


3 


1 


0 


2 


Main 


72 


48 


0 


24 



(N=75) 



15. If you are currently working in an academic library, what is the size of the 
library’s collection? volumes 

Collection size varied from 70 volumes to 5 million volumes. The mean was 598,892.07. 
The median was 200,000. The standard deviation was 1,072,178.23. There were 49 
responses from the paper survey, 40 from the Web survey group notified by email, and 
26 from the Web survey group notified by mail. (N=114) 



16. Do you have your own computer at home? 

17. If yes, do you use it more than once a week? 

18. If you have a computer at home, do you have email access of any sort at 
home? 

19. If yes, do you use it more than once a week? 

20. Do you have Internet access at home? 

21. If yes, do you use it more than once a week? 





Total 

Number of 
Responses 


Paper 

Survey 


Web Survey/ 
Email 


Web Survey/ 
Mail 


Computer at home (N=l 18) 


80 


36 


26 


18 


Use more than once per week 
(N=80) 


63 


28 


20 


15 


Email access at home (N=80) 


65 


29 


20 


16 


Use more than once per week 
(N=68) 


55 


24 


18 


13 


Internet access at home 
(N=80) 


66 


29 


20 


17 


Use more than once per week 
(N=69) 


55 


24 


16 


15 
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22. Do you have your own computer at work? 





Total Number of 
Responses 


Number Yes from 
Paper Survey 


Number Yes from 
Web Survey/Email 


Number Yes from 
Web Survey/Mail 


No 


9 


4 


3 


2 


Yes 


106 


45 


37 


24 



(N=115) 



Questions that appeared on the Web survey only (Ns will be low): 

22. Did you complete this survey at work, at home or at another location? 





Total Number of 
Responses 


Number from 
Web Survey/Email 


Number from 
Web Survey/Mail 


At work 


64 


39 


25 


At home 


3 


1 


2 


At another 
location 


0 


0 


0 



(N=67) 



24, How did you hear about this survey? 





Total Number of 
Responses 


Number from Web 
Survey/ Email 


Number from Web 
Survey/ Mail 


Mail announcement 


25 


0 


25 


Email announcement 


39 


39 


0 


From a colleague 


3 


1 


2 


Other 


0 


0 


0 



(N=67) 



Of the three that indicated they had heard about the survey from a colleague, two 
received the mail notice and one received the email notice (as indicated by the 
information programmed into Gform). 
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